Using Acoustic Signals to Modify Operation of a Lidar System

ABSTRACT

A system includes an acoustic source localization system configured to detect an acoustic signal and determine its location relative to the system. The system also includes a lidar system and a controller. The lidar system includes a light source to emit pulses of light, a scanner to scan a field of regard of the lidar system according to a scan pattern, and a detector to detect light from some of the light pulses scattered by remote targets to generate respective pixels in a point cloud representing objects within the field of regard. The controller configured to receive, from the acoustic source localization system, an indication of the determined location, identify a region of interest within the field of regard that includes the determined location of the source, and cause the lidar system to obtain more data within the region of interest relative to other regions within the field of regard.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to provisional U.S. Application Ser.No. 62/480,349, filed on Mar. 31, 2017, entitled “Improving Operation ofa Lidar System,” the entire disclosure of which is hereby expresslyincorporated by reference herein.

FIELD OF TECHNOLOGY

This disclosure generally relates to lidar systems and, moreparticularly, improving operation of such a system using acousticsignals.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Light detection and ranging (lidar) is a technology that can be used tomeasure distances to remote targets. Typically, a lidar system includesa light source and an optical receiver. The light source can be, forexample, a laser which emits light having a particular operatingwavelength. The operating wavelength of a lidar system may lie, forexample, in the infrared, visible, or ultraviolet portions of theelectromagnetic spectrum. The light source emits light toward a targetwhich then scatters the light. Some of the scattered light is receivedback at the receiver. The system determines the distance to the targetbased on one or more characteristics associated with the returned light.For example, the system may determine the distance to the target basedon the time of flight of a returned pulse of light.

SUMMARY

Generally speaking, a lidar system of this disclosure identifies aregion of interest within its field of regard in view of an acousticsignal whose source is positioned relative to the lidar system usingacoustic source localization techniques. When operating in an autonomousvehicle, the lidar system can collect more data for the region ofinterest when, for example, the sound of a police siren appears to comefrom a certain location within the region of interest.

One example embodiment of the techniques of this disclosure is a systemcomprising an acoustic source localization system, a lidar system, and acontroller communicatively coupled to the acoustic source localizationsystem and the lidar system. The acoustic source localization system isconfigured to detect an acoustic signal and determine a location of asource of the acoustic signal relative to the system. The lidar system alight source configured to emit pulses of light, a scanner configured toscan a field of regard of the lidar system according to a scan pattern,and a detector configured to detect light from some of the light pulsesscattered by one or more remote targets to generate respective pixels ina point cloud representing objects within the field of regard. Thecontroller is configured to (i) receive, from the acoustic sourcelocalization system, an indication of the determined location, (ii)identify a region of interest within the field of regard of the lidarsystem, the region of interest including the determined location of thesource of the acoustic signal, and (iii) cause the lidar system toobtain more data within the region of interest relative to other regionswithin the field of regard.

Another example embodiment of the techniques of this disclosure is amethod for scanning a field of regard of a lidar system. The methodincludes emitting light pulses by a light source in the lidar system;scanning, by a scanner in the lidar system, the field of regard of thelidar system in accordance with a certain scan pattern, includingdirecting the light pulses toward different points within the field ofregard to illuminate a field of view of the light source; receiving anacoustic signal; determining a location of a source of the acousticsignal relative to the lidar system; identifying a region of interestwithin the field of regard based at least in part on the determinedlocation of the source of the sound; modifying at least one operationalparameter of the lidar system to obtain more data within the identifiedregion of interest, relative to other regions within the field ofregard; and detecting light from at least some of the light pulsesscattered by one or more remote targets to generate respective pixels ina point cloud representing objects within the field of regard.

Yet another example embodiment of the techniques of this disclosure isan autonomous vehicle comprising one or more components configured tomaneuver the autonomous vehicle, an acoustic source localization systemconfigured to detect an acoustic signal and determine a location of asource of the acoustic signal relative to the autonomous vehicle, alidar system, and a controller communicatively coupled to the one ormore components configured to maneuver the autonomous vehicle, theacoustic source localization system, and the lidar system. The lidarsystem includes a light source configured to emit pulses of light, ascanner configured to scan a field of regard of the lidar systemaccording to a scan pattern, a detector configured to detect light fromsome of the light pulses scattered by one or more remote targets togenerate respective pixels in a point cloud representing objects withinthe field of regard. The controller is configured to (i) receive, fromthe acoustic source localization system, an indication of the determinedlocation, (ii) identify a region of interest within the field of regardof the lidar system, the region of interest including the determinedlocation of the source of the acoustic signal, (iii) cause the lidarsystem to obtain more data within the region of interest relative toother regions within the field of regard, and (iv) maneuver theautonomous vehicle based at least in part on the point cloud generatedby the lidar system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example light detection and ranging(lidar) system in which the techniques of this disclosure can beimplemented;

FIG. 2 illustrates in more detail several components that can operate inthe system of FIG. 1;

FIG. 3 illustrates an example configuration in which the components ofFIG. 1 scan a 360-degree field of regard through a window in a rotatinghousing;

FIG. 4 illustrates another configuration in which the components of FIG.1 scan a 360-degree field of regard through a substantially transparentstationary housing;

FIG. 5 illustrates an example scan pattern which the lidar system ofFIG. 1 can produce when identifying targets within a field of regard;

FIG. 6 illustrates an example scan pattern which the lidar system ofFIG. 1 can produce when identifying targets within a field of regardusing multiple beams;

FIG. 7 schematically illustrates fields of view (FOVs) of a light sourceand a detector that can operate in the lidar system of FIG. 1;

FIG. 8 illustrates an example configuration of the lidar system of FIG.1 or another suitable lidar system, in which a laser is disposed awayfrom sensor components;

FIG. 9 illustrates an example vehicle in which the lidar system of FIG.1 can operate;

FIG. 10 illustrates an example InGaAs avalanche photodiode which canoperate in the lidar system of FIG. 1;

FIG. 11 illustrates an example photodiode coupled to a pulse-detectioncircuit, which can operate in the lidar system of FIG. 1;

FIG. 12 is a block diagram of an example computing system for generatingcontrol signals for a lidar system based on acoustic signals, which maybe used with the lidar system of FIG. 1;

FIG. 13 illustrates an example region of interest within a field ofregard of a lidar system, which the computing system of FIG. 12 canselect for a certain acoustic signature detected within an acousticsignal;

FIG. 14 illustrates another example region of interest moving across afield of regard of a lidar system, which the computing system of FIG. 12can select for another acoustic signature detected within an acousticsignal; and

FIG. 15 is a flow diagram of an example method for modifying operationof a lidar system in view of acoustic signals, which can be implementedin the computing system of FIG. 12, for example.

DETAILED DESCRIPTION Overview

A lidar system of this disclosure is equipped with an acoustic sourcelocalization system, or communicates with an acoustic sourcelocalization system operating independently of the lidar system in anautonomous vehicle, for example. The acoustic source localization systemcan include a microphone array and a processing module to estimatelocations of currently active sources of acoustic signals and, in somecases, distances to these sources. The acoustic source localizationsystem can be mounted on the roof of the vehicle or built into the frameof the vehicle, for example.

In operation, the acoustic source localization system detects anacoustic signal including amplitudes exceeding a certain threshold andprovides an estimate of a location of the source of this acoustic signalto a controller. Depending on the implementation, the controller can beresponsible for controlling a lidar system only, controlling aperception system including the lidar system, or maneuvering anautonomous vehicle, for example. The acoustic source localization systemin some cases may further analyze the acoustic signal and provide arepresentation of the acoustic signal to the controller in any suitableformat (e.g., as a time-domain description or a frequency-domaindescription). Using the information received from the acoustic sourcelocalization system, the controller can identify a region of interestwithin the field of regard of the lidar system. The lidar system thenmay modify, for a certain period of time, one or more operationalparameters to obtain more data and/or increase the quality of theobtained data for the region of interest relative to other regionswithin the field of regard. These operational parameters can includepulse rate, scan rate, scan pattern, etc. Of course, the lidar systemneed not select regions of interest based solely on indications relatedto sources of acoustic signals, and in general can select regions ofinterest in view of any suitable number of signals.

The controller in some implementations analyzes the detected acousticsignal and determines a likely type of a source of the acoustic signalbased on a set of pre-stored acoustic “signatures” such as a policesiren, another emergency vehicle siren, the screeching of brakes, thescreeching of tires, the sound of collision, the sound of a car horn,etc. The lidar system may modify the one or more operational parametersfurther in view of the determined type of the source of the acousticsignals. For example, the controller may determine that for a certaintype of a source, the zone of interest is likely to move relative to thefield of regard, and that the lidar system should follow the sound forsome time.

In an example scenario, an autonomous vehicle equipped with a lidarsystem of this disclosure and an acoustic source localization systemapproaches an intersection, just as a police car with an activated sirenapproaches the same intersection from the left. The acoustic sourcelocalization system detects an acoustic signal with an amplitude abovethe threshold value (e.g., 90 decibel) and determines an approximatedirection from which the acoustic signal arrived, along with anapproximate distance to the source of this acoustic signal. A controlleroperating in the autonomous vehicle compares the acoustic signal to aset of pre-stored acoustic signatures and determines that the acousticsignal includes the sound of a police siren. The controller retrieves arecord from a local database indicating that the lidar system shoulddouble the information density for a region of interest that spans tendegrees along the horizontal dimension, centered at the current locationof the signal source relative to the autonomous vehicle, and that thelidar system should do so for a period of ten milliseconds. Thecontroller then provides signals to the lidar system specifying thedimensions of the region of interest, the increase in informationdensity (in this case, double) relative to other regions within thefield of regard, and the duration of the period during which theinformation density should be increased.

An example lidar system in which these techniques can be implemented isconsidered next with reference to FIGS. 1-4, followed by a discussion ofthe techniques which the lidar system can implement to scan a field ofregard and generate individual pixels (FIGS. 5-7). An exampleimplementation in a vehicle is then discussed with reference to FIGS. 8and 9. Then, an example photo detector and an example pulse-detectioncircuit are discussed with reference to FIGS. 10 and 11. Exampletechniques for scanning a field of a regard of a lidar system in view ofacoustic signals are discussed with reference to FIGS. 12-15

System Overview

FIG. 1 illustrates an example light detection and ranging (lidar) system100. The lidar system 100 may be referred to as a laser ranging system,a laser radar system, a LIDAR system, a lidar sensor, or a laserdetection and ranging (LADAR or ladar) system. The lidar system 100 mayinclude a light source 110, a mirror 115, a scanner 120, a receiver 140,and a controller 150. The light source 110 may be, for example, a laserwhich emits light having a particular operating wavelength in theinfrared, visible, or ultraviolet portions of the electromagneticspectrum. As a more specific example, the light source 110 may include alaser with an operating wavelength between approximately 1.2 μm and 1.7μm.

In operation, the light source 110 emits an output beam of light 125which may be continuous-wave, pulsed, or modulated in any suitablemanner for a given application. The output beam of light 125 is directeddownrange toward a remote target 130 located a distance D from the lidarsystem 100 and at least partially contained within a field of regard ofthe system 100. Depending on the scenario and/or the implementation ofthe lidar system 100, D can be between 1 m and 1 km, for example.

Once the output beam 125 reaches the downrange target 130, the target130 may scatter or, in some cases, reflect at least a portion of lightfrom the output beam 125, and some of the scattered or reflected lightmay return toward the lidar system 100. In the example of FIG. 1, thescattered or reflected light is represented by input beam 135, whichpasses through the scanner 120, which may be referred to as a beamscanner, optical scanner, or laser scanner. The input beam 135 passesthrough the scanner 120 to the mirror 115, which may be referred to asan overlap mirror, superposition mirror, or beam-combiner mirror. Themirror 115 in turn directs the input beam 135 to the receiver 140. Theinput 135 may contain only a relatively small fraction of the light fromthe output beam 125. For example, the ratio of average power, peakpower, or pulse energy of the input beam 135 to average power, peakpower, or pulse energy of the output beam 125 may be approximately 10⁻¹,10⁻², 10⁻³, 10⁻⁴, 10⁻⁵, 10⁻⁶, 10⁻⁷, 10⁻⁸, 10⁻⁹, 10⁻¹⁰, 10⁻¹¹, or 10⁻¹².As another example, if a pulse of the output beam 125 has a pulse energyof 1 microjoule (μJ), then the pulse energy of a corresponding pulse ofthe input beam 135 may have a pulse energy of approximately 10nanojoules (nJ), 1 nJ, 100 picojoules (pJ), 10 pJ, 1 pJ, 100 femtojoules(fJ), 10 fJ, 1 fJ, 100 attojoules (aJ), 10 aJ, or 1 aJ.

The output beam 125 may be referred to as a laser beam, light beam,optical beam, emitted beam, or just beam; and the input beam 135 may bereferred to as a return beam, received beam, return light, receivedlight, input light, scattered light, or reflected light. As used herein,scattered light may refer to light that is scattered or reflected by thetarget 130. The input beam 135 may include light from the output beam125 that is scattered by the target 130, light from the output beam 125that is reflected by the target 130, or a combination of scattered andreflected light from target 130.

The operating wavelength of a lidar system 100 may lie, for example, inthe infrared, visible, or ultraviolet portions of the electromagneticspectrum. The Sun also produces light in these wavelength ranges, andthus sunlight can act as background noise which can obscure signal lightdetected by the lidar system 100. This solar background noise can resultin false-positive detections or can otherwise corrupt measurements ofthe lidar system 100, especially when the receiver 140 includes SPADdetectors (which can be highly sensitive).

Generally speaking, the light from the Sun that passes through theEarth's atmosphere and reaches a terrestrial-based lidar system such asthe system 100 can establish an optical background noise floor for thissystem. Thus, in order for a signal from the lidar system 100 to bedetectable, the signal must rise above the background noise floor. It isgenerally possible to increase the signal-to-noise (SNR) ratio of thelidar system 100 by raising the power level of the output beam 125, butin some situations it may be desirable to keep the power level of theoutput beam 125 relatively low. For example, increasing transmit powerlevels of the output beam 125 can result in the lidar system 100 notbeing eye-safe.

In some implementations, the lidar system 100 operates at one or morewavelengths between approximately 1400 nm and approximately 1600 nm. Forexample, the light source 110 may produce light at approximately 1550nm.

In some implementations, the lidar system 100 operates at frequencies atwhich atmospheric absorption is relatively low. For example, the lidarsystem 100 can operate at wavelengths in the approximate ranges from 980nm to 1110 nm or from 1165 nm to 1400 nm.

In other implementations, the lidar system 100 operates at frequenciesat which atmospheric absorption is high. For example, the lidar system100 can operate at wavelengths in the approximate ranges from 930 nm to980 nm, from 1100 nm to 1165 nm, or from 1400 nm to 1460 nm.

According to some implementations, the lidar system 100 can include aneye-safe laser, or the lidar system 100 can be classified as an eye-safelaser system or laser product. An eye-safe laser, laser system, or laserproduct may refer to a system with an emission wavelength, averagepower, peak power, peak intensity, pulse energy, beam size, beamdivergence, exposure time, or scanned output beam such that emittedlight from the system presents little or no possibility of causingdamage to a person's eyes. For example, the light source 110 or lidarsystem 100 may be classified as a Class 1 laser product (as specified bythe 60825-1 standard of the International Electrotechnical Commission(IEC)) or a Class I laser product (as specified by Title 21, Section1040.10 of the United States Code of Federal Regulations (CFR)) that issafe under all conditions of normal use. In some implementations, thelidar system 100 may be classified as an eye-safe laser product (e.g.,with a Class 1 or Class I classification) configured to operate at anysuitable wavelength between approximately 1400 nm and approximately 2100nm. In some implementations, the light source 110 may include a laserwith an operating wavelength between approximately 1400 nm andapproximately 1600 nm, and the lidar system 100 may be operated in aneye-safe manner. In some implementations, the light source 110 or thelidar system 100 may be an eye-safe laser product that includes ascanned laser with an operating wavelength between approximately 1530 nmand approximately 1560 nm. In some implementations, the lidar system 100may be a Class 1 or Class I laser product that includes a fiber laser orsolid-state laser with an operating wavelength between approximately1400 nm and approximately 1600 nm.

The receiver 140 may receive or detect photons from the input beam 135and generate one or more representative signals. For example, thereceiver 140 may generate an output electrical signal 145 that isrepresentative of the input beam 135. The receiver may send theelectrical signal 145 to the controller 150. Depending on theimplementation, the controller 150 may include one or more processors,an application-specific integrated circuit (ASIC), a field-programmablegate array (FPGA), and/or other suitable circuitry configured to analyzeone or more characteristics of the electrical signal 145 to determineone or more characteristics of the target 130, such as its distancedownrange from the lidar system 100. More particularly, the controller150 may analyze the time of flight or phase modulation for the beam oflight 125 transmitted by the light source 110. If the lidar system 100measures a time of flight of T (e.g., T represents a round-trip time offlight for an emitted pulse of light to travel from the lidar system 100to the target 130 and back to the lidar system 100), then the distance Dfrom the target 130 to the lidar system 100 may be expressed as D=c·T/2,where c is the speed of light (approximately 3.0×10⁸ m/s).

As a more specific example, if the lidar system 100 measures the time offlight to be T=300 ns, then the lidar system 100 can determine thedistance from the target 130 to the lidar system 100 to be approximatelyD=45.0 m. As another example, the lidar system 100 measures the time offlight to be T=1.33 μs and accordingly determines that the distance fromthe target 130 to the lidar system 100 is approximately D=199.5 m. Thedistance D from lidar system 100 to the target 130 may be referred to asa distance, depth, or range of the target 130. As used herein, the speedof light c refers to the speed of light in any suitable medium, such asfor example in air, water, or vacuum. The speed of light in vacuum isapproximately 2.9979×10⁸ m/s, and the speed of light in air (which has arefractive index of approximately 1.0003) is approximately 2.9970×10⁸m/s.

The target 130 may be located a distance D from the lidar system 100that is less than or equal to a maximum range R_(MAX) of the lidarsystem 100. The maximum range R_(MAX) (which also may be referred to asa maximum distance) of a lidar system 100 may correspond to the maximumdistance over which the lidar system 100 is configured to sense oridentify targets that appear in a field of regard of the lidar system100. The maximum range of lidar system 100 may be any suitable distance,such as for example, 25 m, 50 m, 100 m, 200 m, 500 m, or 1 km. As aspecific example, a lidar system with a 200-m maximum range may beconfigured to sense or identify various targets located up to 200 maway. For a lidar system with a 200-m maximum range (R_(MAX)=200 m), thetime of flight corresponding to the maximum range is approximately2·R_(MAX)/c≅1.33 μs.

In some implementations, the light source 110, the scanner 120, and thereceiver 140 may be packaged together within a single housing 155, whichmay be a box, case, or enclosure that holds or contains all or part of alidar system 100. The housing 155 includes a window 157 through whichthe beams 125 and 135 pass. In one example implementation, thelidar-system housing 155 contains the light source 110, the overlapmirror 115, the scanner 120, and the receiver 140 of a lidar system 100.The controller 150 may reside within the same housing 155 as thecomponents 110, 120, and 140, or the controller 150 may reside remotelyfrom the housing.

Moreover, in some implementations, the housing 155 includes multiplelidar sensors, each including a respective scanner and a receiver.Depending on the particular implementation, each of the multiple sensorscan include a separate light source or a common light source. Themultiple sensors can be configured to cover non-overlapping adjacentfields of regard or partially overlapping fields of regard, depending onthe implementation.

The housing 155 may be an airtight or watertight structure that preventswater vapor, liquid water, dirt, dust, or other contaminants fromgetting inside the housing 155. The housing 155 may be filled with a dryor inert gas, such as for example dry air, nitrogen, or argon. Thehousing 155 may include one or more electrical connections for conveyingelectrical power or electrical signals to and/or from the housing.

The window 157 may be made from any suitable substrate material, such asfor example, glass or plastic (e.g., polycarbonate, acrylic,cyclic-olefin polymer, or cyclic-olefin copolymer). The window 157 mayinclude an interior surface (surface A) and an exterior surface (surfaceB), and surface A or surface B may include a dielectric coating havingparticular reflectivity values at particular wavelengths. A dielectriccoating (which may be referred to as a thin-film coating, interferencecoating, or coating) may include one or more thin-film layers ofdielectric materials (e.g., SiO₂, TiO₂, Al₂O₃, Ta₂O₅, MgF₂, LaF₃, orAlF₃) having particular thicknesses (e.g., thickness less than 1 μm) andparticular refractive indices. A dielectric coating may be depositedonto surface A or surface B of the window 157 using any suitabledeposition technique, such as for example, sputtering or electron-beamdeposition.

The dielectric coating may have a high reflectivity at a particularwavelength or a low reflectivity at a particular wavelength. Ahigh-reflectivity (HR) dielectric coating may have any suitablereflectivity value (e.g., a reflectivity greater than or equal to 80%,90%, 95%, or 99%) at any suitable wavelength or combination ofwavelengths. A low-reflectivity dielectric coating (which may bereferred to as an anti-reflection (AR) coating) may have any suitablereflectivity value (e.g., a reflectivity less than or equal to 5%, 2%,1%, 0.5%, or 0.2%) at any suitable wavelength or combination ofwavelengths. In particular embodiments, a dielectric coating may be adichroic coating with a particular combination of high or lowreflectivity values at particular wavelengths. For example, a dichroiccoating may have a reflectivity of less than or equal to 0.5% atapproximately 1550-1560 nm and a reflectivity of greater than or equalto 90% at approximately 800-1500 nm.

In some implementations, surface A or surface B has a dielectric coatingthat is anti-reflecting at an operating wavelength of one or more lightsources 110 contained within enclosure 155. An AR coating on surface Aand surface B may increase the amount of light at an operatingwavelength of light source 110 that is transmitted through the window157. Additionally, an AR coating at an operating wavelength of the lightsource 110 may reduce the amount of incident light from output beam 125that is reflected by the window 157 back into the housing 155. In anexample implementation, each of surface A and surface B has an ARcoating with reflectivity less than 0.5% at an operating wavelength oflight source 110. As an example, if the light source 110 has anoperating wavelength of approximately 1550 nm, then surface A andsurface B may each have an AR coating with a reflectivity that is lessthan 0.5% from approximately 1547 nm to approximately 1553 nm. Inanother implementation, each of surface A and surface B has an ARcoating with reflectivity less than 1% at the operating wavelengths ofthe light source 110. For example, if the housing 155 encloses twosensor heads with respective light sources, the first light source emitspulses at a wavelength of approximately 1535 nm and the second lightsource emits pulses at a wavelength of approximately 1540 nm, thensurface A and surface B may each have an AR coating with reflectivityless than 1% from approximately 1530 nm to approximately 1545 nm.

The window 157 may have an optical transmission that is greater than anysuitable value for one or more wavelengths of one or more light sources110 contained within the housing 155. As an example, the window 157 mayhave an optical transmission of greater than or equal to 70%, 80%, 90%,95%, or 99% at a wavelength of light source 110. In one exampleimplementation, the window 157 can transmit greater than or equal to 95%of light at an operating wavelength of the light source 110. In anotherimplementation, the window 157 transmits greater than or equal to 90% oflight at the operating wavelengths of the light sources enclosed withinthe housing 155.

Surface A or surface B may have a dichroic coating that isanti-reflecting at one or more operating wavelengths of one or morelight sources 110 and high-reflecting at wavelengths away from the oneor more operating wavelengths. For example, surface A may have an ARcoating for an operating wavelength of the light source 110, and surfaceB may have a dichroic coating that is AR at the light-source operatingwavelength and HR for wavelengths away from the operating wavelength. Acoating that is HR for wavelengths away from a light-source operatingwavelength may prevent most incoming light at unwanted wavelengths frombeing transmitted through the window 117. In one implementation, iflight source 110 emits optical pulses with a wavelength of approximately1550 nm, then surface A may have an AR coating with a reflectivity ofless than or equal to 0.5% from approximately 1546 nm to approximately1554 nm. Additionally, surface B may have a dichroic coating that is ARat approximately 1546-1554 nm and HR (e.g., reflectivity of greater thanor equal to 90%) at approximately 800-1500 nm and approximately1580-1700 nm.

Surface B of the window 157 may include a coating that is oleophobic,hydrophobic, or hydrophilic. A coating that is oleophobic (or,lipophobic) may repel oils (e.g., fingerprint oil or other non-polarmaterial) from the exterior surface (surface B) of the window 157. Acoating that is hydrophobic may repel water from the exterior surface.For example, surface B may be coated with a material that is botholeophobic and hydrophobic. A coating that is hydrophilic attracts waterso that water may tend to wet and form a film on the hydrophilic surface(rather than forming beads of water as may occur on a hydrophobicsurface). If surface B has a hydrophilic coating, then water (e.g., fromrain) that lands on surface B may form a film on the surface. Thesurface film of water may result in less distortion, deflection, orocclusion of an output beam 125 than a surface with a non-hydrophiliccoating or a hydrophobic coating.

With continued reference to FIG. 1, the light source 110 may include apulsed laser configured to produce or emit pulses of light with acertain pulse duration. In an example implementation, the pulse durationor pulse width of the pulsed laser is approximately 10 picoseconds (ps)to 20 nanoseconds (ns). In another implementation, the light source 110is a pulsed laser that produces pulses with a pulse duration ofapproximately 1-4 ns. In yet another implementation, the light source110 is a pulsed laser that produces pulses at a pulse repetitionfrequency of approximately 100 kHz to 5 MHz or a pulse period (e.g., atime between consecutive pulses) of approximately 200 ns to 10 μs. Thelight source 110 may have a substantially constant or a variable pulserepetition frequency, depending on the implementation. As an example,the light source 110 may be a pulsed laser that produces pulses at asubstantially constant pulse repetition frequency of approximately 640kHz (e.g., 640,000 pulses per second), corresponding to a pulse periodof approximately 1.56 μs. As another example, the light source 110 mayhave a pulse repetition frequency that can be varied from approximately500 kHz to 3 MHz. As used herein, a pulse of light may be referred to asan optical pulse, a light pulse, or a pulse, and a pulse repetitionfrequency may be referred to as a pulse rate.

In general, the output beam 125 may have any suitable average opticalpower, and the output beam 125 may include optical pulses with anysuitable pulse energy or peak optical power. Some examples of theaverage power of the output beam 125 include the approximate values of 1mW, 10 mW, 100 mW, 1 W, and 10 W. Example values of pulse energy of theoutput beam 125 include the approximate values of 0.1 μJ, 1 μJ, 10 μJ,100 μJ, and 1 mJ. Examples of peak power values of pulses included inthe output beam 125 are the approximate values of 10 W, 100 W, 1 kW, 5kW, 10 kW. An example optical pulse with a duration of 1 ns and a pulseenergy of 1 μJ has a peak power of approximately 1 kW. If the pulserepetition frequency is 500 kHz, then the average power of the outputbeam 125 with 1-μJ pulses is approximately 0.5 W, in this example.

The light source 110 may include a laser diode, such as a Fabry-Perotlaser diode, a quantum well laser, a distributed Bragg reflector (DBR)laser, a distributed feedback (DFB) laser, or a vertical-cavitysurface-emitting laser (VCSEL). The laser diode operating in the lightsource 110 may be an aluminum-gallium-arsenide (AlGaAs) laser diode, anindium-gallium-arsenide (InGaAs) laser diode, or anindium-gallium-arsenide-phosphide (InGaAsP) laser diode, or any othersuitable diode. In some implementations, the light source 110 includes apulsed laser diode with a peak emission wavelength of approximately1400-1600 nm. Further, the light source 110 may include a laser diodethat is current-modulated to produce optical pulses.

In some implementation, the light source 110 includes a pulsed laserdiode followed by one or more optical-amplification stages. For example,the light source 110 may be a fiber-laser module that includes acurrent-modulated laser diode with a peak wavelength of approximately1550 nm, followed by a single-stage or a multi-stage erbium-doped fiberamplifier (EDFA). As another example, the light source 110 may include acontinuous-wave (CW) or quasi-CW laser diode followed by an externaloptical modulator (e.g., an electro-optic modulator), and the output ofthe modulator may be fed into an optical amplifier. In yet otherimplementations, the light source 110 may include a pulsed solid-statelaser or a pulsed fiber laser.

In some implementations, the output beam of light 125 emitted by thelight source 110 is a collimated optical beam with any suitable beamdivergence, such as a divergence of approximately 0.1 to 3.0 milliradian(mrad). Divergence of the output beam 125 may refer to an angularmeasure of an increase in beam size (e.g., a beam radius or beamdiameter) as the output beam 125 travels away from the light source 110or the lidar system 100. The output beam 125 may have a substantiallycircular cross section with a beam divergence characterized by a singledivergence value. For example, the output beam 125 with a circular crosssection and a divergence of 1 mrad may have a beam diameter or spot sizeof approximately 10 cm at a distance of 100 m from the lidar system 100.In some implementations, the output beam 125 may be an astigmatic beamor may have a substantially elliptical cross section and may becharacterized by two divergence values. As an example, the output beam125 may have a fast axis and a slow axis, where the fast-axis divergenceis greater than the slow-axis divergence. As another example, the outputbeam 125 may be an astigmatic beam with a fast-axis divergence of 2 mradand a slow-axis divergence of 0.5 mrad.

The output beam of light 125 emitted by light source 110 may beunpolarized or randomly polarized, may have no specific or fixedpolarization (e.g., the polarization may vary with time), or may have aparticular polarization (e.g., the output beam 125 may be linearlypolarized, elliptically polarized, or circularly polarized). As anexample, the light source 110 may produce linearly polarized light, andthe lidar system 100 may include a quarter-wave plate that converts thislinearly polarized light into circularly polarized light. The lidarsystem 100 may transmit the circularly polarized light as the outputbeam 125, and receive the input beam 135, which may be substantially orat least partially circularly polarized in the same manner as the outputbeam 125 (e.g., if the output beam 125 is right-hand circularlypolarized, then the input beam 135 may also be right-hand circularlypolarized). The input beam 135 may pass through the same quarter-waveplate (or a different quarter-wave plate), resulting in the input beam135 being converted to linearly polarized light which is orthogonallypolarized (e.g., polarized at a right angle) with respect to thelinearly polarized light produced by light source 110. As anotherexample, the lidar system 100 may employ polarization-diversitydetection where two polarization components are detected separately. Theoutput beam 125 may be linearly polarized, and the lidar system 100 maysplit the input beam 135 into two polarization components (e.g.,s-polarization and p-polarization) which are detected separately by twophotodiodes (e.g., a balanced photoreceiver that includes twophotodiodes).

With continued reference to FIG. 1, the output beam 125 and input beam135 may be substantially coaxial. In other words, the output beam 125and input beam 135 may at least partially overlap or share a commonpropagation axis, so that the input beam 135 and the output beam 125travel along substantially the same optical path (albeit in oppositedirections). As the lidar system 100 scans the output beam 125 across afield of regard, the input beam 135 may follow along with the outputbeam 125, so that the coaxial relationship between the two beams ismaintained.

The lidar system 100 also may include one or more optical componentsconfigured to condition, shape, filter, modify, steer, or direct theoutput beam 125 and/or the input beam 135. For example, lidar system 100may include one or more lenses, mirrors, filters (e.g., bandpass orinterference filters), beam splitters, polarizers, polarizing beamsplitters, wave plates (e.g., half-wave or quarter-wave plates),diffractive elements, or holographic elements. In some implementations,lidar system 100 includes a telescope, one or more lenses, or one ormore mirrors to expand, focus, or collimate the output beam 125 to adesired beam diameter or divergence. As an example, the lidar system 100may include one or more lenses to focus the input beam 135 onto anactive region of the receiver 140. As another example, the lidar system100 may include one or more flat mirrors or curved mirrors (e.g.,concave, convex, or parabolic mirrors) to steer or focus the output beam125 or the input beam 135. For example, the lidar system 100 may includean off-axis parabolic mirror to focus the input beam 135 onto an activeregion of receiver 140. As illustrated in FIG. 1, the lidar system 100may include the mirror 115, which may be a metallic or dielectricmirror. The mirror 115 may be configured so that the light beam 125passes through the mirror 115. As an example, mirror 115 may include ahole, slot, or aperture through which the output light beam 125 passes.As another example, the mirror 115 may be configured so that at least80% of the output beam 125 passes through the mirror 115 and at least80% of the input beam 135 is reflected by the mirror 115. In someimplementations, the mirror 115 may provide for the output beam 125 andthe input beam 135 to be substantially coaxial, so that the beams 125and 135 travel along substantially the same optical path, in oppositedirections.

Generally speaking, the scanner 120 steers the output beam 125 in one ormore directions downrange. The scanner 120 may include one or morescanning mirrors and one or more actuators driving the mirrors torotate, tilt, pivot, or move the mirrors in an angular manner about oneor more axes, for example. For example, the first mirror of the scannermay scan the output beam 125 along a first direction, and the secondmirror may scan the output beam 125 along a second direction that issubstantially orthogonal to the first direction. Example implementationsof the scanner 120 are discussed in more detail below with reference toFIG. 2.

The scanner 120 may be configured to scan the output beam 125 over a5-degree angular range, 20-degree angular range, 30-degree angularrange, 60-degree angular range, or any other suitable angular range. Forexample, a scanning mirror may be configured to periodically rotate overa 15-degree range, which results in the output beam 125 scanning acrossa 30-degree range (e.g., a 0-degree rotation by a scanning mirrorresults in a 20-degree angular scan of the output beam 125). A field ofregard (FOR) of the lidar system 100 may refer to an area, region, orangular range over which the lidar system 100 may be configured to scanor capture distance information. When the lidar system 100 scans theoutput beam 125 within a 30-degree scanning range, the lidar system 100may be referred to as having a 30-degree angular field of regard. Asanother example, a lidar system 100 with a scanning mirror that rotatesover a 30-degree range may produce the output beam 125 that scans acrossa 60-degree range (e.g., a 60-degree FOR). In various implementations,the lidar system 100 may have a FOR of approximately 10°, 20°, 40°, 60°,120°, or any other suitable FOR. The FOR also may be referred to as ascan region.

The scanner 120 may be configured to scan the output beam 125horizontally and vertically, and the lidar system 100 may have aparticular FOR along the horizontal direction and another particular FORalong the vertical direction. For example, the lidar system 100 may havea horizontal FOR of 10° to 120° and a vertical FOR of 2° to 45°.

The one or more scanning mirrors of the scanner 120 may becommunicatively coupled to the controller 150 which may control thescanning mirror(s) so as to guide the output beam 125 in a desireddirection downrange or along a desired scan pattern. In general, a scanpattern may refer to a pattern or path along which the output beam 125is directed, and also may be referred to as an optical scan pattern,optical scan path, or scan path. As an example, the scanner 120 mayinclude two scanning mirrors configured to scan the output beam 125across a 60° horizontal FOR and a 20° vertical FOR. The two scannermirrors may be controlled to follow a scan path that substantiallycovers the 60°×20° FOR. The lidar system 100 can use the scan path togenerate a point cloud with pixels that substantially cover the 60°×20°FOR. The pixels may be approximately evenly distributed across the60°×20° FOR. Alternately, the pixels may have a particular non-uniformdistribution (e.g., the pixels may be distributed across all or aportion of the 60°×20° FOR, and the pixels may have a higher density inone or more particular regions of the 60°×20° FOR).

In operation, the light source 110 may emit pulses of light which thescanner 120 scans across a FOR of lidar system 100. The target 130 mayscatter one or more of the emitted pulses, and the receiver 140 maydetect at least a portion of the pulses of light scattered by the target130.

The receiver 140 may be referred to as (or may include) a photoreceiver,optical receiver, optical sensor, detector, photodetector, or opticaldetector. The receiver 140 in some implementations receives or detectsat least a portion of the input beam 135 and produces an electricalsignal that corresponds to the input beam 135. For example, if the inputbeam 135 includes an optical pulse, then the receiver 140 may produce anelectrical current or voltage pulse that corresponds to the opticalpulse detected by the receiver 140. In an example implementation, thereceiver 140 includes one or more avalanche photodiodes (APDs) or one ormore single-photon avalanche diodes (SPADs). In another implementation,the receiver 140 includes one or more PN photodiodes (e.g., a photodiodestructure formed by a p-type semiconductor and a n-type semiconductor)or one or more PIN photodiodes (e.g., a photodiode structure formed byan undoped intrinsic semiconductor region located between p-type andn-type regions).

The receiver 140 may have an active region or anavalanche-multiplication region that includes silicon, germanium, orInGaAs. The active region of receiver 140 may have any suitable size,such as for example, a diameter or width of approximately 50-500 μm. Thereceiver 140 may include circuitry that performs signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection. For example, thereceiver 140 may include a transimpedance amplifier that converts areceived photocurrent (e.g., a current produced by an APD in response toa received optical signal) into a voltage signal. The receiver 140 maydirect the voltage signal to pulse-detection circuitry that produces ananalog or digital output signal 145 that corresponds to one or morecharacteristics (e.g., rising edge, falling edge, amplitude, orduration) of a received optical pulse. For example, the pulse-detectioncircuitry may perform a time-to-digital conversion to produce a digitaloutput signal 145. The receiver 140 may send the electrical outputsignal 145 to the controller 150 for processing or analysis, e.g., todetermine a time-of-flight value corresponding to a received opticalpulse.

The controller 150 may be electrically coupled or otherwisecommunicatively coupled to one or more of the light source 110, thescanner 120, and the receiver 140. The controller 150 may receiveelectrical trigger pulses or edges from the light source 110, where eachpulse or edge corresponds to the emission of an optical pulse by thelight source 110. The controller 150 may provide instructions, a controlsignal, or a trigger signal to the light source 110 indicating when thelight source 110 should produce optical pulses. For example, thecontroller 150 may send an electrical trigger signal that includeselectrical pulses, where the light source 110 emits an optical pulse inresponse to each electrical pulse. Further, the controller 150 may causethe light source 110 to adjust one or more of the frequency, period,duration, pulse energy, peak power, average power, or wavelength of theoptical pulses produced by light source 110.

The controller 150 may determine a time-of-flight value for an opticalpulse based on timing information associated with when the pulse wasemitted by light source 110 and when a portion of the pulse (e.g., theinput beam 135) was detected or received by the receiver 140. Thecontroller 150 may include circuitry that performs signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection.

As indicated above, the lidar system 100 may be used to determine thedistance to one or more downrange targets 130. By scanning the lidarsystem 100 across a field of regard, the system can be used to map thedistance to a number of points within the field of regard. Each of thesedepth-mapped points may be referred to as a pixel or a voxel. Acollection of pixels captured in succession (which may be referred to asa depth map, a point cloud, or a frame) may be rendered as an image ormay be analyzed to identify or detect objects or to determine a shape ordistance of objects within the FOR. For example, a depth map may cover afield of regard that extends 60° horizontally and 15° vertically, andthe depth map may include a frame of 100-2000 pixels in the horizontaldirection by 4-400 pixels in the vertical direction.

The lidar system 100 may be configured to repeatedly capture or generatepoint clouds of a field of regard at any suitable frame rate betweenapproximately 0.1 frames per second (FPS) and approximately 1,000 FPS.For example, the lidar system 100 may generate point clouds at a framerate of approximately 0.1 FPS, 0.5 FPS, 1 FPS, 2 FPS, 5 FPS, 10 FPS, 20FPS, 100 FPS, 500 FPS, or 1,000 FPS. In an example implementation, thelidar system 100 is configured to produce optical pulses at a rate of5×10⁵ pulses/second (e.g., the system may determine 500,000 pixeldistances per second) and scan a frame of 1000×50 pixels (e.g., 50,000pixels/frame), which corresponds to a point-cloud frame rate of 10frames per second (e.g., 10 point clouds per second). The point-cloudframe rate may be substantially fixed or dynamically adjustable,depending on the implementation. For example, the lidar system 100 maycapture one or more point clouds at a particular frame rate (e.g., 1 Hz)and then switch to capture one or more point clouds at a different framerate (e.g., 10 Hz). In general, the lidar system can use a slower framerate (e.g., 1 Hz) to capture one or more high-resolution point clouds,and use a faster frame rate (e.g., 10 Hz) to rapidly capture multiplelower-resolution point clouds.

The field of regard of the lidar system 100 can overlap, encompass, orenclose at least a portion of the target 130, which may include all orpart of an object that is moving or stationary relative to lidar system100. For example, the target 130 may include all or a portion of aperson, vehicle, motorcycle, truck, train, bicycle, wheelchair,pedestrian, animal, road sign, traffic light, lane marking, road-surfacemarking, parking space, pylon, guard rail, traffic barrier, pothole,railroad crossing, obstacle in or near a road, curb, stopped vehicle onor beside a road, utility pole, house, building, trash can, mailbox,tree, any other suitable object, or any suitable combination of all orpart of two or more objects.

With continued reference to FIG. 1, audio sensors 159A-C may be disposedin any suitable location on the vehicle to detect acoustic signals, sothat a processing module (discussed below) can locate the sources of theacoustic signals. In the depicted example implementation, the audiosensors 159A and 159B are disposed on two sides of the housing 155, andthe audio sensor 159 is disposed above or below the housing 155. Inanother implementation (illustrated in FIG. 9, for example), the audiosensors are organized into a microphone array enclosed within a singlehousing unit, which can be mounted on the roof of the vehicle, insidethe vehicle on the dashboard, on the frame of the vehicle, etc. In anycase, the audio sensors 159A-C may operate as components of an acousticsource localization system that estimates the location of a signalsource using triangulation or another suitable technique.

Now referring to FIG. 2, a scanner 162 and a receiver 164 can operate inthe lidar system of FIG. 1 as the scanner 120 and the receiver 140,respectively. More generally, the scanner 162 and the receiver 164 canoperate in any suitable lidar system.

The scanner 162 may include any suitable number of mirrors driven by anysuitable number of mechanical actuators. For example, the scanner 162may include a galvanometer scanner, a resonant scanner, a piezoelectricactuator, a polygonal scanner, a rotating-prism scanner, a voice coilmotor, a DC motor, a brushless DC motor, a stepper motor, or amicroelectromechanical systems (MEMS) device, or any other suitableactuator or mechanism.

A galvanometer scanner (which also may be referred to as a galvanometeractuator) may include a galvanometer-based scanning motor with a magnetand coil. When an electrical current is supplied to the coil, arotational force is applied to the magnet, which causes a mirrorattached to the galvanometer scanner to rotate. The electrical currentsupplied to the coil may be controlled to dynamically change theposition of the galvanometer mirror. A resonant scanner (which may bereferred to as a resonant actuator) may include a spring-like mechanismdriven by an actuator to produce a periodic oscillation at asubstantially fixed frequency (e.g., 1 kHz). A MEMS-based scanningdevice may include a mirror with a diameter between approximately 1 and10 mm, where the mirror is rotated using electromagnetic orelectrostatic actuation. A voice coil motor (which may be referred to asa voice coil actuator) may include a magnet and coil. When an electricalcurrent is supplied to the coil, a translational force is applied to themagnet, which causes a mirror attached to the magnet to move or rotate.

In an example implementation, the scanner 162 includes a single mirrorconfigured to scan an output beam 170 along a single direction (e.g.,the scanner 162 may be a one-dimensional scanner that scans along ahorizontal or vertical direction). The mirror may be a flat scanningmirror attached to a scanner actuator or mechanism which scans themirror over a particular angular range. The mirror may be driven by oneactuator (e.g., a galvanometer) or two actuators configured to drive themirror in a push-pull configuration. When two actuators drive the mirrorin one direction in a push-pull configuration, the actuators may belocated at opposite ends or sides of the mirror. The actuators mayoperate in a cooperative manner so that when one actuator pushes on themirror, the other actuator pulls on the mirror, and vice versa. Inanother example implementation, two voice coil actuators arranged in apush-pull configuration drive a mirror along a horizontal or verticaldirection.

In some implementations, the scanner 162 may include one mirrorconfigured to be scanned along two axes, where two actuators arranged ina push-pull configuration provide motion along each axis. For example,two resonant actuators arranged in a horizontal push-pull configurationmay drive the mirror along a horizontal direction, and another pair ofresonant actuators arranged in a vertical push-pull configuration maydrive mirror along a vertical direction. In another exampleimplementation, two actuators scan the output beam 170 along twodirections (e.g., horizontal and vertical), where each actuator providesrotational motion along a particular direction or about a particularaxis.

The scanner 162 also may include one mirror driven by two actuatorsconfigured to scan the mirror along two substantially orthogonaldirections. For example, a resonant actuator or a galvanometer actuatormay drive one mirror along a substantially horizontal direction, and agalvanometer actuator may drive the mirror along a substantiallyvertical direction. As another example, two resonant actuators may drivea mirror along two substantially orthogonal directions.

In some implementations, the scanner 162 includes two mirrors, where onemirror scans the output beam 170 along a substantially horizontaldirection and the other mirror scans the output beam 170 along asubstantially vertical direction. In the example of FIG. 2, the scanner162 includes two mirrors, a mirror 180-1 and a mirror 180-2. The mirror180-1 may scan the output beam 170 along a substantially horizontaldirection, and the mirror 180-2 may scan the output beam 170 along asubstantially vertical direction (or vice versa). Mirror 180-1 or mirror180-2 may be a flat mirror, a curved mirror, or a polygon mirror withtwo or more reflective surfaces.

The scanner 162 in other implementations includes two galvanometerscanners driving respective mirrors. For example, the scanner 162 mayinclude a galvanometer actuator that scans the mirror 180-1 along afirst direction (e.g., vertical), and the scanner 162 may includeanother galvanometer actuator that scans the mirror 180-2 along a seconddirection (e.g., horizontal). In yet another implementation, the scanner162 includes two mirrors, where a galvanometer actuator drives onemirror, and a resonant actuator drives the other mirror. For example, agalvanometer actuator may scan the mirror 180-1 along a first direction,and a resonant actuator may scan the mirror 180-2 along a seconddirection. The first and second scanning directions may be substantiallyorthogonal to one another, e.g., the first direction may besubstantially vertical, and the second direction may be substantiallyhorizontal. In yet another implementation, the scanner 162 includes twomirrors, where one mirror is a polygon mirror that is rotated in onedirection (e.g., clockwise or counter-clockwise) by an electric motor(e.g., a brushless DC motor). For example, mirror 180-1 may be a polygonmirror that scans the output beam 170 along a substantially horizontaldirection, and mirror 180-2 may scan the output beam 170 along asubstantially vertical direction. A polygon mirror may have two or morereflective surfaces, and the polygon mirror may be continuously rotatedin one direction so that the output beam 170 is reflected sequentiallyfrom each of the reflective surfaces. A polygon mirror may have across-sectional shape that corresponds to a polygon, where each side ofthe polygon has a reflective surface. For example, a polygon mirror witha square cross-sectional shape may have four reflective surfaces, and apolygon mirror with a pentagonal cross-sectional shape may have fivereflective surfaces.

To direct the output beam 170 along a particular scan pattern, thescanner 162 may include two or more actuators driving a single mirrorsynchronously. For example, the two or more actuators can drive themirror synchronously along two substantially orthogonal directions tomake the output beam 170 follow a scan pattern with substantiallystraight lines. In some implementations, the scanner 162 may include twomirrors and actuators driving the two mirrors synchronously to generatea scan pattern that includes substantially straight lines. For example,a galvanometer actuator may drive the mirror 180-2 with a substantiallylinear back-and-forth motion (e.g., the galvanometer may be driven witha substantially sinusoidal or triangle-shaped waveform) that causes theoutput beam 170 to trace a substantially horizontal back-and-forthpattern, and another galvanometer actuator may scan the mirror 180-1along a substantially vertical direction. The two galvanometers may besynchronized so that for every 64 horizontal traces, the output beam 170makes a single trace along a vertical direction. Whether one or twomirrors are used, the substantially straight lines can be directedsubstantially horizontally, vertically, or along any other suitabledirection.

The scanner 162 also may apply a dynamically adjusted deflection along avertical direction (e.g., with a galvanometer actuator) as the outputbeam 170 is scanned along a substantially horizontal direction (e.g.,with a galvanometer or resonant actuator) to achieve the straight lines.If a vertical deflection is not applied, the output beam 170 may traceout a curved path as it scans from side to side. In someimplementations, the scanner 162 uses a vertical actuator to apply adynamically adjusted vertical deflection as the output beam 170 isscanned horizontally as well as a discrete vertical offset between eachhorizontal scan (e.g., to step the output beam 170 to a subsequent rowof a scan pattern).

With continued reference to FIG. 2, an overlap mirror 190 in thisexample implementation is configured to overlap the input beam 172 andoutput beam 170, so that the beams 170 and 172 are substantiallycoaxial. In FIG. 2, the overlap mirror 190 includes a hole, slot, oraperture 192 through which the output beam 170 passes, and a reflectingsurface 194 that reflects at least a portion of the input beam 172toward the receiver 164. The overlap mirror 190 may be oriented so thatinput beam 172 and output beam 170 are at least partially overlapped.

In some implementations, the overlap mirror 190 may not include a hole192. For example, the output beam 170 may be directed to pass by a sideof mirror 190 rather than passing through an aperture 192. The outputbeam 170 may pass alongside mirror 190 and may be oriented at a slightangle with respect to the orientation of the input beam 172. As anotherexample, the overlap mirror 190 may include a small reflective sectionconfigured to reflect the output beam 170, and the rest of the overlapmirror 190 may have an AR coating configured to transmit the input beam172.

The input beam 172 may pass through a lens 196 which focuses the beamonto an active region 166 of the receiver 164. The active region 166 mayrefer to an area over which receiver 164 may receive or detect inputlight. The active region may have any suitable size or diameter d, suchas for example, a diameter of approximately 25 μm, 50 μm, 80 μm, 100 μm,200 μm, 500 μm, 1 mm, 2 mm, or 5 mm. The overlap mirror 190 may have areflecting surface 194 that is substantially flat or the reflectingsurface 194 may be curved (e.g., the mirror 190 may be an off-axisparabolic mirror configured to focus the input beam 172 onto an activeregion of the receiver 140).

The aperture 192 may have any suitable size or diameter Φ₁, and theinput beam 172 may have any suitable size or diameter Φ₂, where Φ₂ isgreater than Φ₁. For example, the aperture 192 may have a diameter Φ₁ ofapproximately 0.2 mm, 0.5 mm, 1 mm, 2 mm, 3 mm, 5 mm, or 10 mm, and theinput beam 172 may have a diameter Φ₂ of approximately 2 mm, 5 mm, 10mm, 15 mm, 20 mm, 30 mm, 40 mm, or 50 mm. In some implementations, thereflective surface 194 of the overlap mirror 190 may reflect 70% or moreof input beam 172 toward the receiver 164. For example, if thereflective surface 194 has a reflectivity R at an operating wavelengthof the light source 160, then the fraction of input beam 172 directedtoward the receiver 164 may be expressed as R×[1−(Φ₁/Φ₂)²]. As a morespecific example, if R is 95%, Φ₁ is 2 mm, and Φ₂ is 10 mm, thenapproximately 91% of the input beam 172 may be directed toward thereceiver 164 by the reflective surface 194.

FIG. 3 illustrates an example configuration in which several componentsof the lidar system 100 may operate to scan a 360-degree view of regard.Generally speaking, the field of view of a light source in thisconfiguration follows a circular trajectory and accordingly defines acircular scan pattern on a two-dimensional plane. All points on thetrajectory remain at the same elevation relative to the ground level,according to one implementation. In this case, separate beams may followthe circular trajectory with certain vertical offsets relative to eachother. In another implementation, the points of the trajectory maydefine a spiral scan pattern in three-dimensional space. A single beamcan be sufficient to trace out the spiral scan pattern but, if desired,multiple beams can be used.

In the example of FIG. 3, a rotating scan module 200 revolves around acentral axis in one or both directions as indicated. An electric motormay drive the rotating scan module 200 around the central axis at aconstant speed, for example. The rotating scan module 200 includes ascanner, a receiver, an overlap mirror, etc. The components of therotating module 200 may be similar to the scanner 120, the receiver 140,and the overlap mirror 115. In some implementations, the subsystem 200also includes a light source and a controller. In other implementations,the light source and/or the controller are disposed apart from therotating scan module 200 and/or exchange optical and electrical signalswith the components of the rotating scan module 200 via correspondinglinks.

The rotating scan module 200 may include a housing 210 with a window212. Similar to the window 157 of FIG. 1, the window 212 may be made ofglass, plastic, or any other suitable material. The window 212 allowsoutbound beams as well as return signals to pass through the housing210. The arc length defined by the window 212 can correspond to anysuitable percentage of the circumference of the housing 210. Forexample, the arc length can correspond to 5%, 20%, 30%, 60%, or possiblyeven 100% of the circumference.

Now referring to FIG. 4, a rotating scan module 220 is generally similarto the rotating scan module 200. In this implementation, however, thecomponents of the rotating scan module 220 are disposed on a platform222 which rotates inside a stationary circular housing 230. In thisimplementation, the circular housing 230 is substantially transparent tolight at the lidar-system operating wavelength to pass inbound andoutbound light signals. The circular housing 230 in a sense defines acircular window similar to the window 212, and may be made of similarmaterial.

In some implementations, a lidar system may include or may be referredto as a flash lidar system, a staring lidar system, or a time-of-flightcamera. A flash lidar system may include a pulsed light source and areceiver. A flash lidar system may not include a scanner, and the lightsource of a flash lidar system may not be scanned across the field ofregard. The receiver may include multiple detectors, and the flash lidarsystem may capture a point cloud with a single emitted pulse of lightthat illuminates a field of regard.

Generating Pixels Within a Field of Regard of the Lidar System

FIG. 5 illustrates an example scan pattern 240 which the lidar system100 of FIG. 1 can produce. The lidar system 100 may be configured toscan output optical beam 125 along one or more scan patterns 240. Insome implementations, the scan pattern 240 corresponds to a scan acrossany suitable field of regard (FOR) having any suitable horizontal FOR(FOR_(H)) and any suitable vertical FOR (FOR_(V)). For example, acertain scan pattern may have a field of regard represented by angulardimensions (e.g., FOR_(H)×FOR_(V)) 40°×30°, 90°×40°, or 60°×15°. Asanother example, a certain scan pattern may have a FOR_(II) greater thanor equal to 10°, 25°, 30°, 40°, 60°, 90°, or 120°. As yet anotherexample, a certain scan pattern may have a FOR_(V) greater than or equalto 2°, 5°, 10°, 15°, 20°, 30°, or 45°. In the example of FIG. 5,reference line 246 represents a center of the field of regard of scanpattern 240. The reference line 246 may have any suitable orientation,such as, a horizontal angle of 0° (e.g., reference line 246 may beoriented straight ahead) and a vertical angle of 0° (e.g., referenceline 246 may have an inclination of 0°), or the reference line 246 mayhave a nonzero horizontal angle or a nonzero inclination (e.g., avertical angle of +10° or −10°). In FIG. 5, if the scan pattern 240 hasa 60°×15° field of regard, then the scan pattern 240 covers a ±30°horizontal range with respect to reference line 246 and a ±7.5° verticalrange with respect to reference line 246. Additionally, the optical beam125 in FIG. 5 has an orientation of approximately −15° horizontal and+3° vertical with respect to reference line 246. The beam 125 may bereferred to as having an azimuth of −15° and an altitude of +3° relativeto the reference line 246. An azimuth (which may be referred to as anazimuth angle) may represent a horizontal angle with respect to thereference line 246, and an altitude (which may be referred to as analtitude angle, elevation, or elevation angle) may represent a verticalangle with respect to the reference line 246.

The scan pattern 240 may include multiple pixels 242, and each pixel 242may be associated with one or more laser pulses and one or morecorresponding distance measurements. A cycle of scan pattern 240 mayinclude a total of P_(x)×P_(y) pixels 242 (e.g., a two-dimensionaldistribution of P_(x) by P_(y) pixels). For example, the scan pattern240 may include a distribution with dimensions of approximately100-2,000 pixels 242 along a horizontal direction and approximately4-400 pixels 242 along a vertical direction. As another example, thescan pattern 240 may include a distribution of 1,000 pixels 242 alongthe horizontal direction by 64 pixels 242 along the vertical direction(e.g., the frame size is 1000×64 pixels) for a total of 64,000 pixelsper cycle of scan pattern 240. The number of pixels 242 along ahorizontal direction may be referred to as a horizontal resolution ofthe scan pattern 240, and the number of pixels 242 along a verticaldirection may be referred to as a vertical resolution of the scanpattern 240. As an example, the scan pattern 240 may have a horizontalresolution of greater than or equal to 100 pixels 242 and a verticalresolution of greater than or equal to 4 pixels 242. As another example,the scan pattern 240 may have a horizontal resolution of 100-2,000pixels 242 and a vertical resolution of 4-400 pixels 242.

Each pixel 242 may be associated with a distance (e.g., a distance to aportion of a target 130 from which the corresponding laser pulse wasscattered) or one or more angular values. As an example, the pixel 242may be associated with a distance value and two angular values (e.g., anazimuth and altitude) that represent the angular location of the pixel242 with respect to the lidar system 100. A distance to a portion of thetarget 130 may be determined based at least in part on a time-of-flightmeasurement for a corresponding pulse. An angular value (e.g., anazimuth or altitude) may correspond to an angle (e.g., relative toreference line 246) of the output beam 125 (e.g., when a correspondingpulse is emitted from lidar system 100) or an angle of the input beam135 (e.g., when an input signal is received by lidar system 100). Insome implementations, the lidar system 100 determines an angular valuebased at least in part on a position of a component of the scanner 120.For example, an azimuth or altitude value associated with the pixel 242may be determined from an angular position of one or more correspondingscanning mirrors of the scanner 120.

In some implementations, the lidar system 100 concurrently directsmultiple beams across the field of regard. In the example implementationof FIG. 6, the lidar system generates output beams 250A, 250B, 250C, . .. 250N etc., each of which follows a linear scan pattern 254A, 254B,254C, . . . 254N. The number of parallel lines can be 2, 4, 12, 20, orany other suitable number. The lidar system 100 may angularly separatethe beams 250A, 250B, 250C, . . . 250N, so that, for example, theseparation between beams 250A and 250B at a certain distance may be 30cm, and the separation between the same beams 250A and 250B at a longerdistance may be 50 cm.

Similar to the scan pattern 240, each of the linear scan patterns 254A-Nincludes pixels associated with one or more laser pulses and distancemeasurements. FIG. 6 illustrates example pixels 252A, 252B and 252Calong the scan patterns 254A, 254B and 254C, respectively. The lidarsystem 100 in this example may generate the values for the pixels252A-252N at the same time, thus increasing the rate at which values forpixels are determined.

Depending on the implementation, the lidar system 100 may output thebeams 250A-N at the same wavelength or different wavelengths. The beam250A for example may have the wavelength of 1540 nm, the beam 250B mayhave the wavelength of 1550 nm, the beam 250C may have the wavelength of1560 nm, etc. The number of different wavelengths the lidar system 100uses need not match the number of beams. Thus, the lidar system 100 inthe example implementation of FIG. 6 may use M wavelengths with N beams,where 1≤M≤N.

Next, FIG. 7 illustrates an example light-source field of view (FOV_(L))and receiver field of view (FOV_(R)) for the lidar system 100. The lightsource 110 may emit pulses of light as the FOV_(L) and FOV_(R) arescanned by the scanner 120 across a field of regard (FOR). Thelight-source field of view may refer to an angular cone illuminated bythe light source 110 at a particular instant of time. Similarly, areceiver field of view may refer to an angular cone over which thereceiver 140 may receive or detect light at a particular instant oftime, and any light outside the receiver field of view may not bereceived or detected. For example, as the scanner 120 scans thelight-source field of view across a field of regard, the lidar system100 may send the pulse of light in the direction the FOV_(L) is pointingat the time the light source 110 emits the pulse. The pulse of light mayscatter off the target 130, and the receiver 140 may receive and detecta portion of the scattered light that is directed along or containedwithin the FOV_(R).

In some implementations, the scanner 120 is configured to scan both alight-source field of view and a receiver field of view across a fieldof regard of the lidar system 100. The lidar system 100 may emit anddetect multiple pulses of light as the scanner 120 scans the FOV_(L) andFOV_(R) across the field of regard while tracing out the scan pattern240. The scanner 120 in some implementations scans the light-sourcefield of view and the receiver field of view synchronously with respectto one another. In this case, as the scanner 120 scans FOV_(L) across ascan pattern 240, the FOV_(R) follows substantially the same path at thesame scanning speed. Additionally, the FOV_(L) and FOV_(R) may maintainthe same relative position to one another as the scanner 120 scansFOV_(L) and FOV_(R) across the field of regard. For example, the FOV_(L)may be substantially overlapped with or centered inside the FOV_(R) (asillustrated in FIG. 7), and the scanner 120 may maintain this relativepositioning between FOV_(L) and FOV_(R) throughout a scan. As anotherexample, the FOV_(R) may lag behind the FOV_(L) by a particular, fixedamount throughout a scan (e.g., the FOV_(R) may be offset from theFOV_(L) in a direction opposite the scan direction).

The FOV_(L) may have an angular size or extent Θ_(L) that issubstantially the same as or that corresponds to the divergence of theoutput beam 125, and the FOV_(R) may have an angular size or extentΘ_(R) that corresponds to an angle over which the receiver 140 mayreceive and detect light. The receiver field of view may be any suitablesize relative to the light-source field of view. For example, thereceiver field of view may be smaller than, substantially the same sizeas, or larger than the angular extent of the light-source field of view.In some implementations, the light-source field of view has an angularextent of less than or equal to 50 milliradians, and the receiver fieldof view has an angular extent of less than or equal to 50 milliradians.The FOV_(L) may have any suitable angular extent Θ_(L), such as forexample, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. Similarly,the FOV_(R) may have any suitable angular extent Θ_(R), such as forexample, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. Thelight-source field of view and the receiver field of view may haveapproximately equal angular extents. As an example, Θ_(L) and Θ_(R) mayboth be approximately equal to 1 mrad, 2 mrad, or 3 mrad. In someimplementations, the receiver field of view is larger than thelight-source field of view, or the light-source field of view is largerthan the receiver field of view. For example, Θ_(L) may be approximatelyequal to 1.5 mrad, and Θ_(R) may be approximately equal to 3 mrad.

A pixel 242 may represent or correspond to a light-source field of view.As the output beam 125 propagates from the light source 110, thediameter of the output beam 125 (as well as the size of thecorresponding pixel 242) may increase according to the beam divergenceΘ_(L). As an example, if the output beam 125 has a Θ_(L) of 2 mrad, thenat a distance of 100 m from the lidar system 100, the output beam 125may have a size or diameter of approximately 20 cm, and a correspondingpixel 242 may also have a corresponding size or diameter ofapproximately 20 cm. At a distance of 200 m from the lidar system 100,the output beam 125 and the corresponding pixel 242 may each have adiameter of approximately 40 cm.

A Lidar System Operating in a Vehicle

As indicated above, one or more lidar systems 100 may be integrated intoa vehicle. In one example implementation, multiple lidar systems 100 maybe integrated into a car to provide a complete 360-degree horizontal FORaround the car. As another example, 4-10 lidar systems 100, each systemhaving a 45-degree to 90-degree horizontal FOR, may be combined togetherto form a sensing system that provides a point cloud covering a360-degree horizontal FOR. The lidar systems 100 may be oriented so thatadjacent FORs have an amount of spatial or angular overlap to allow datafrom the multiple lidar systems 100 to be combined or stitched togetherto form a single or continuous 360-degree point cloud. As an example,the FOR of each lidar system 100 may have approximately 1-15 degrees ofoverlap with an adjacent FOR. In particular embodiments, a vehicle mayrefer to a mobile machine configured to transport people or cargo. Forexample, a vehicle may include, may take the form of, or may be referredto as a car, automobile, motor vehicle, truck, bus, van, trailer,off-road vehicle, farm vehicle, lawn mower, construction equipment, golfcart, motorhome, taxi, motorcycle, scooter, bicycle, skateboard, train,snowmobile, watercraft (e.g., a ship or boat), aircraft (e.g., afixed-wing aircraft, helicopter, or dirigible), or spacecraft. Inparticular embodiments, a vehicle may include an internal combustionengine or an electric motor that provides propulsion for the vehicle.

In some implementations, one or more lidar systems 100 are included in avehicle as part of an advanced driver assistance system (ADAS) to assista driver of the vehicle in the driving process. For example, a lidarsystem 100 may be part of an ADAS that provides information or feedbackto a driver (e.g., to alert the driver to potential problems or hazards)or that automatically takes control of part of a vehicle (e.g., abraking system or a steering system) to avoid collisions or accidents.The lidar system 100 may be part of a vehicle ADAS that providesadaptive cruise control, automated braking, automated parking, collisionavoidance, alerts the driver to hazards or other vehicles, maintains thevehicle in the correct lane, or provides a warning if an object oranother vehicle is in a blind spot.

In some cases, one or more lidar systems 100 are integrated into avehicle as part of an autonomous-vehicle driving system. In an exampleimplementation, the lidar system 100 provides information about thesurrounding environment to a driving system of an autonomous vehicle. Anautonomous-vehicle driving system may include one or more computingsystems that receive information from the lidar system 100 about thesurrounding environment, analyze the received information, and providecontrol signals to the vehicle's driving systems (e.g., steering wheel,accelerator, brake, or turn signal). For example, the lidar system 100integrated into an autonomous vehicle may provide an autonomous-vehicledriving system with a point cloud every 0.1 seconds (e.g., the pointcloud has a 10 Hz update rate, representing 10 frames per second). Theautonomous-vehicle driving system may analyze the received point cloudsto sense or identify targets 130 and their respective locations,distances, or speeds, and the autonomous-vehicle driving system mayupdate control signals based on this information. As an example, if thelidar system 100 detects a vehicle ahead that is slowing down orstopping, the autonomous-vehicle driving system may send instructions torelease the accelerator and apply the brakes.

An autonomous vehicle may be referred to as an autonomous car,driverless car, self-driving car, robotic car, or unmanned vehicle. Anautonomous vehicle may be a vehicle configured to sense its environmentand navigate or drive with little or no human input. For example, anautonomous vehicle may be configured to drive to any suitable locationand control or perform all safety-critical functions (e.g., driving,steering, braking, parking) for the entire trip, with the driver notexpected to control the vehicle at any time. As another example, anautonomous vehicle may allow a driver to safely turn their attentionaway from driving tasks in particular environments (e.g., on freeways),or an autonomous vehicle may provide control of a vehicle in all but afew environments, requiring little or no input or attention from thedriver.

An autonomous vehicle may be configured to drive with a driver presentin the vehicle, or an autonomous vehicle may be configured to operatethe vehicle with no driver present. As an example, an autonomous vehiclemay include a driver's seat with associated controls (e.g., steeringwheel, accelerator pedal, and brake pedal), and the vehicle may beconfigured to drive with no one seated in the driver's seat or withlittle or no input from a person seated in the driver's seat. As anotherexample, an autonomous vehicle may not include any driver's seat orassociated driver's controls, and the vehicle may perform substantiallyall driving functions (e.g., driving, steering, braking, parking, andnavigating) without human input. As another example, an autonomousvehicle may be configured to operate without a driver (e.g., the vehiclemay be configured to transport human passengers or cargo without adriver present in the vehicle). As another example, an autonomousvehicle may be configured to operate without any human passengers (e.g.,the vehicle may be configured for transportation of cargo without havingany human passengers onboard the vehicle).

In some implementations, a light source of a lidar system is locatedremotely from some of the other components of the lidar system such asthe scanner and the receiver. Moreover, a lidar system implemented in avehicle may include fewer light sources than scanners and receivers.

FIG. 8 illustrates an example configuration in which a laser-sensor link320 includes an optical link 330 and an electrical link 350 coupledbetween a laser 300 and a sensor 310. The laser 300 may be configured toemit pulses of light and may be referred to as a laser system, laserhead, or light source. The laser 300 may include, may be part of, may besimilar to, or may be substantially the same as the light source 110illustrated in FIG. 1 and discussed above. Further, the scanner 302, thereceiver 304, the controller 306, and the mirror 308 may be similar tothe scanner 120, the receiver 140, the controller 150, and the mirror115 discussed above. In the example of FIG. 8, the laser 300 is coupledto the remotely located sensor 310 by a laser-sensor link 320 (which maybe referred to as a link). The sensor 310 may be referred to as a sensorhead and may include the mirror 308, the scanner 302, the receiver 304,and the controller 306. In an example implementation, the laser 300includes a pulsed laser diode (e.g., a pulsed DFB laser) followed by anoptical amplifier, and light from the laser 300 is conveyed by anoptical fiber of the laser-sensor link 320 of a suitable length to thescanner 120 in a remotely located sensor 310.

The laser-sensor link 320 may include any suitable number of opticallinks 330 (e.g., 0, 1, 2, 3, 5, or 10) and any suitable number ofelectrical links 350 (e.g., 0, 1, 2, 3, 5, or 10). In the exampleconfiguration depicted in FIG. 8, the laser-sensor link 320 includes oneoptical link 330 from the laser 300 to an output collimator 340 and oneelectrical link 350 that connects the laser 300 to the controller 150.The optical link 330 may include optical fiber (which may be referred toas fiber-optic cable or fiber) that conveys, carries, transports, ortransmits light between the laser 300 and the sensor 310. The opticalfiber may be, for example, single-mode (SM) fiber, multi-mode (MM)fiber, large-mode-area (LMA) fiber, polarization-maintaining (PM) fiber,photonic-crystal or photonic-bandgap fiber, gain fiber (e.g.,rare-earth-doped optical fiber for use in an optical amplifier), or anysuitable combination thereof. The output collimator 340 receives opticalpulses conveyed from the laser 300 by the optical link 330 and producesa free-space optical beam 312 that includes the optical pulses. Theoutput collimator 340 directs the free-space optical beam 312 throughthe mirror 308 and to the scanner 302.

The electrical link 350 may include electrical wire or cable (e.g., acoaxial cable or twisted-pair cable) that conveys or transmitselectrical power and/or one or more electrical signals between the laser300 and the sensor 310. For example, the laser 300 may include a powersupply or a power conditioner that provides electrical power to thelaser 300, and additionally, the power supply or power conditioner mayprovide power to one or more components of the sensor 310 (e.g., thescanner 304, the receiver 304, and/or the controller 306) via the one ormore electrical links 350. The electrical link 350 in someimplementations may convey electrical signals that include data orinformation in analog or digital format. Further, the electrical link350 may provide an interlock signal from the sensor 310 to the laser300. If the controller 306 detects a fault condition indicating aproblem with the sensor 310 or the overall lidar system, the controller306 may change a voltage on the interlock line (e.g., from 5 V to 0 V)indicating that the laser 300 should shut down, stop emitting light, orreduce the power or energy of emitted light. A fault condition may betriggered by a failure of the scanner 302, a failure of the receiver304, or by a person or object coming within a threshold distance of thesensor 310 (e.g., within 0.1 m, 0.5 m, 1 m, 5 m, or any other suitabledistance).

As discussed above, a lidar system can include one or more processors todetermine a distance D to a target. In the implementation illustrated inFIG. 8, the controller 306 may be located in the laser 300 or in thesensor 310, or parts of the controller 150 may be distributed betweenthe laser 300 and the sensor 310. In an example implementation, eachsensor head 310 of a lidar system includes electronics (e.g., anelectronic filter, transimpedance amplifier, threshold detector, ortime-to-digital (TDC) converter) configured to receive or process asignal from the receiver 304 or from an APD or SPAD of the receiver 304.Additionally, the laser 300 may include processing electronicsconfigured to determine a time-of-flight value or a distance to thetarget based on a signal received from the sensor head 310 via theelectrical link 350.

Next, FIG. 9 illustrates an example vehicle 354 with a lidar system 351that includes a laser 352 with multiple sensor heads 360 coupled to thelaser 352 via multiple laser-sensor links 370. The laser 352 and thesensor heads 360 may be similar to the laser 300 and the sensor 310discussed above, in some implementations. For example, each of thelaser-sensor links 370 may include one or more optical links and/or oneor more electrical links. The sensor heads 360 in FIG. 9 are positionedor oriented to provide a greater than 30-degree view of an environmentaround the vehicle. More generally, a lidar system with multiple sensorheads may provide a horizontal field of regard around a vehicle ofapproximately 30°, 45°, 60°, 90°, 120°, 180°, 270°, or 360°. Each of thesensor heads may be attached to or incorporated into a bumper, fender,grill, side panel, spoiler, roof, headlight assembly, taillightassembly, rear-view mirror assembly, hood, trunk, window, or any othersuitable part of the vehicle.

In the example of FIG. 9, four sensor heads 360 are positioned at ornear the four corners of the vehicle (e.g., the sensor heads may beincorporated into a light assembly, side panel, bumper, or fender), andthe laser 352 may be located within the vehicle (e.g., in or near thetrunk). The four sensor heads 360 may each provide a 90° to 120°horizontal field of regard (FOR), and the four sensor heads 360 may beoriented so that together they provide a complete 360-degree view aroundthe vehicle. As another example, the lidar system 351 may include sixsensor heads 360 positioned on or around a vehicle, where each of thesensor heads 360 provides a 60° to 90° horizontal FOR. As anotherexample, the lidar system 351 may include eight sensor heads 360, andeach of the sensor heads 360 may provide a 45° to 60° horizontal FOR. Asyet another example, the lidar system 351 may include six sensor heads360, where each of the sensor heads 360 provides a 70° horizontal FORwith an overlap between adjacent FORs of approximately 10°. As anotherexample, the lidar system 351 may include two sensor heads 360 whichtogether provide a forward-facing horizontal FOR of greater than orequal to 30°.

Data from each of the sensor heads 360 may be combined or stitchedtogether to generate a point cloud that covers a greater than or equalto 30-degree horizontal view around a vehicle. For example, the laser352 may include a controller or processor that receives data from eachof the sensor heads 360 (e.g., via a corresponding electrical link 370)and processes the received data to construct a point cloud covering a360-degree horizontal view around a vehicle or to determine distances toone or more targets. The point cloud or information from the point cloudmay be provided to a vehicle controller 372 via a correspondingelectrical, optical, or radio link 370. In some implementations, thepoint cloud is generated by combining data from each of the multiplesensor heads 360 at a controller included within the laser 352 andprovided to the vehicle controller 372. In other implementations, eachof the sensor heads 360 includes a controller or process that constructsa point cloud for a portion of the 360-degree horizontal view around thevehicle and provides the respective point cloud to the vehiclecontroller 372. The vehicle controller 372 then combines or stitchestogether the points clouds from the respective sensor heads 360 toconstruct a combined point cloud covering a 360-degree horizontal view.Still further, the vehicle controller 372 in some implementationscommunicates with a remote server to process point cloud data.

In any event, the vehicle 354 may be an autonomous vehicle where thevehicle controller 372 provides control signals to various components390 within the vehicle 354 to maneuver and otherwise control operationof the vehicle 354. The components 390 are depicted in an expanded viewin FIG. 9 for ease of illustration only. The components 390 may includean accelerator 374, brakes 376, a vehicle engine 378, a steeringmechanism 380, lights 382 such as brake lights, head lights, reverselights, emergency lights, etc., a gear selector 384, and/or othersuitable components that effectuate and control movement of the vehicle354. The gear selector 384 may include the park, reverse, neutral, drivegears, etc. Each of the components 390 may include an interface viawhich the component receives commands from the vehicle controller 372such as “increase speed,” “decrease speed,” “turn left 5 degrees,”“activate left turn signal,” etc. and, in some cases, provides feedbackto the vehicle controller 372.

With continued reference to FIG. 9, a microphone array 392 may beinstalled on the roof, below the roof, or in any other suitable locationon the vehicle 354. The microphone array 392 may be communicativelycoupled to the vehicle controller 372 via a wired or wirelesscommunication link 394. In some implementations, the microphone array392 also may be connected to the sensor heads 360 or a controller of thelidar system that includes the sensor heads 360. The microphone array392 may be a component in an acoustic source localization system thatalso includes a processing module that analyzes acoustic signalsreceived by the microphone array 392. Processing of acoustic signals isdiscussed further below with reference to FIGS. 12-15.

In some implementations, the vehicle controller 372 receives point clouddata from the laser 352 or sensor heads 360 via the link 370 andanalyzes the received point cloud data to sense or identify targets 130and their respective locations, distances, speeds, shapes, sizes, typeof target (e.g., vehicle, human, tree, animal), etc. The vehiclecontroller 372 then provides control signals via the link 370 to thecomponents 390 to control operation of the vehicle based on the analyzedinformation. For example, the vehicle controller 372 may identify anintersection based on the point cloud data and determine that theintersection is the appropriate location at which to make a left turn.Accordingly, the vehicle controller 372 may provide control signals tothe steering mechanism 380, the accelerator 374, and brakes 376 formaking a proper left turn. In another example, the vehicle controller372 may identify a traffic light based on the point cloud data anddetermine that the vehicle 354 needs to come to a stop. As a result, thevehicle controller 372 may provide control signals to release theaccelerator 374 and apply the brakes 376.

Example Receiver Implementation

FIG. 10 illustrates an example InGaAs avalanche photodiode (APD) 400.Referring back to FIG. 1, the receiver 140 may include one or more APDs400 configured to receive and detect light from input light such as thebeam 135. More generally, the APD 400 can operate in any suitablereceiver of input light. The APD 400 may be configured to detect aportion of pulses of light which are scattered by a target locateddownrange from the lidar system in which the APD 400 operates. Forexample, the APD 400 may receive a portion of a pulse of light scatteredby the target 130 depicted in FIG. 1, and generate an electrical-currentsignal corresponding to the received pulse of light.

The APD 400 may include doped or undoped layers of any suitablesemiconductor material, such as for example, silicon, germanium, InGaAs,InGaAsP, or indium phosphide (InP). Additionally, the APD 400 mayinclude an upper electrode 402 and a lower electrode 406 for couplingthe ADP 400 to an electrical circuit. The APD 400 for example may beelectrically coupled to a voltage source that supplies a reverse-biasvoltage V to the APD 400. Additionally, the APD 400 may be electricallycoupled to a transimpedance amplifier which receives electrical currentgenerated by the APD 400 and produces an output voltage signal thatcorresponds to the received current. The upper electrode 402 or lowerelectrode 406 may include any suitable electrically conductive material,such as for example a metal (e.g., gold, copper, silver, or aluminum), atransparent conductive oxide (e.g., indium tin oxide), a carbon-nanotubematerial, or polysilicon. In some implementations, the upper electrode402 is partially transparent or has an opening to allow input light 410to pass through to the active region of the APD 400. In FIG. 10, theupper electrode 402 may have a ring shape that at least partiallysurrounds the active region of the APD 400, where the active regionrefers to an area over which the APD 400 may receive and detect theinput light 410. The active region may have any suitable size ordiameter d, such as for example, a diameter of approximately 25 μm, 50μm, 80 μm, 100 μm, 200 μm, 500 μm, 1 mm, 2 mm, or 5 mm.

The APD 400 may include any suitable combination of any suitablesemiconductor layers having any suitable doping (e.g., n-doped, p-doped,or intrinsic undoped material). In the example of FIG. 10, the InGaAsAPD 400 includes a p-doped InP layer 420, an InP avalanche layer 422, anabsorption layer 424 with n-doped InGaAs or InGaAsP, and an n-doped InPsubstrate layer 426. Depending on the implementation, the APD 400 mayinclude separate absorption and avalanche layers, or a single layer mayact as both an absorption and avalanche region. The APD 400 may operateelectrically as a PN diode or a PIN diode, and, during operation, theAPD 400 may be reverse-biased with a positive voltage V applied to thelower electrode 406 with respect to the upper electrode 402. The appliedreverse-bias voltage V may have any suitable value, such as for exampleapproximately 5 V, 10 V, 20 V, 30 V, 50 V, 75 V, 100 V, or 200 V.

In FIG. 10, photons of the input light 410 may be absorbed primarily inthe absorption layer 424, resulting in the generation of electron-holepairs (which may be referred to as photo-generated carriers). Forexample, the absorption layer 424 may be configured to absorb photonscorresponding to the operating wavelength of the lidar system 100 (e.g.,any suitable wavelength between approximately 1400 nm and approximately1600 nm). In the avalanche layer 422, an avalanche-multiplicationprocess occurs where carriers (e.g., electrons or holes) generated inthe absorption layer 424 collide with the semiconductor lattice of theabsorption layer 424, and produce additional carriers through impactionization. This avalanche process can repeat numerous times so that onephoto-generated carrier may result in the generation of multiplecarriers. As an example, a single photon absorbed in the absorptionlayer 424 may lead to the generation of approximately 10, 50, 100, 200,500, 1000, 10,000, or any other suitable number of carriers through anavalanche-multiplication process. The carriers generated in an APD 400may produce an electrical current that is coupled to an electricalcircuit which may perform signal amplification, sampling, filtering,signal conditioning, analog-to-digital conversion, time-to-digitalconversion, pulse detection, threshold detection, rising-edge detection,or falling-edge detection.

The number of carriers generated from a single photo-generated carriermay increase as the applied reverse bias V is increased. If the appliedreverse bias V is increased above a particular value referred to as theAPD breakdown voltage, then a single carrier can trigger aself-sustaining avalanche process (e.g., the output of the APD 400 issaturated regardless of the input light level). The APD 400 that isoperated at or above a breakdown voltage may be referred to as asingle-photon avalanche diode (SPAD) and may be referred to as operatingin a Geiger mode or a photon-counting mode. The APD 400 that is operatedbelow a breakdown voltage may be referred to as a linear APD, and theoutput current generated by the APD 400 may be sent to an amplifiercircuit (e.g., a transimpedance amplifier). The receiver 140 (seeFIG. 1) may include an APD configured to operate as a SPAD and aquenching circuit configured to reduce a reverse-bias voltage applied tothe SPAD when an avalanche event occurs in the SPAD. The APD 400configured to operate as a SPAD may be coupled to an electronicquenching circuit that reduces the applied voltage V below the breakdownvoltage when an avalanche-detection event occurs. Reducing the appliedvoltage may halt the avalanche process, and the applied reverse-biasvoltage may then be re-set to await a subsequent avalanche event.Additionally, the APD 400 may be coupled to a circuit that generates anelectrical output pulse or edge when an avalanche event occurs.

In some implementations, the APD 400 or the APD 400 along withtransimpedance amplifier have a noise-equivalent power (NEP) that isless than or equal to 100 photons, 50 photons, 30 photons, 20 photons,or 10 photons. For example, the APD 400 may be operated as a SPAD andmay have a NEP of less than or equal to 20 photons. As another example,the APD 400 may be coupled to a transimpedance amplifier that producesan output voltage signal with a NEP of less than or equal to 50 photons.The NEP of the APD 400 is a metric that quantifies the sensitivity ofthe APD 400 in terms of a minimum signal (or a minimum number ofphotons) that the APD 400 can detect. The NEP may correspond to anoptical power (or to a number of photons) that results in asignal-to-noise ratio of 1, or the NEP may represent a threshold numberof photons above which an optical signal may be detected. For example,if the APD 400 has a NEP of 20 photons, then the input beam 410 with 20photons may be detected with a signal-to-noise ratio of approximately 1(e.g., the APD 400 may receive 20 photons from the input beam 410 andgenerate an electrical signal representing the input beam 410 that has asignal-to-noise ratio of approximately 1). Similarly, the input beam 410with 100 photons may be detected with a signal-to-noise ratio ofapproximately 5. In some implementations, the lidar system 100 with theAPD 400 (or a combination of the APD 400 and transimpedance amplifier)having a NEP of less than or equal to 100 photons, 50 photons, 30photons, 20 photons, or 10 photons offers improved detection sensitivitywith respect to a conventional lidar system that uses a PN or PINphotodiode. For example, an InGaAs PIN photodiode used in a conventionallidar system may have a NEP of approximately 10⁴ to 10⁵ photons, and thenoise level in a lidar system with an InGaAs PIN photodiode may be 10³to 10⁴ times greater than the noise level in a lidar system 100 with theInGaAs APD detector 400.

Referring back to FIG. 1, an optical filter may be located in front ofthe receiver 140 and configured to transmit light at one or moreoperating wavelengths of the light source 110 and attenuate light atsurrounding wavelengths. For example, an optical filter may be afree-space spectral filter located in front of APD 400 of FIG. 10. Thisspectral filter may transmit light at the operating wavelength of thelight source 110 (e.g., between approximately 1530 nm and 1560 nm) andattenuate light outside that wavelength range. As a more specificexample, light with wavelengths of approximately 400-1530 nm or1560-2000 nm may be attenuated by any suitable amount, such as forexample, by at least 5 dB, 10 dB, 20 dB, 30 dB, or 40 dB.

Next, FIG. 11 illustrates an APD 502 coupled to an examplepulse-detection circuit 504. The APD 502 can be similar to the APD 400discussed above with reference to FIG. 10, or can be any other suitabledetector. The pulse-detection circuit 504 can operate in the lidarsystem of FIG. 1 as part of the receiver 140. Further, thepulse-detection circuit 504 can operate in the receiver 164 of FIG. 2,the receiver 304 of FIG. 8, or any other suitable receiver. Thepulse-detection circuit 504 alternatively can be implemented in thecontroller 150, the controller 306, or another suitable controller. Insome implementations, parts of the pulse-detection circuit 504 canoperate in a receiver and other parts of the pulse-detection circuit 504can operate in a controller. For example, components 510 and 512 may bea part of the receiver 140, and components 514 and 516 may be a part ofthe controller 150.

The pulse-detection circuit 504 may include circuitry that receives asignal from a detector (e.g., an electrical current from the APD 502)and performs current-to-voltage conversion, signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection. The pulse-detectioncircuit 504 may determine whether an optical pulse has been received bythe APD 502 or may determine a time associated with receipt of anoptical pulse by the APD 502. Additionally, the pulse-detection circuit504 may determine a duration of a received optical pulse. In an exampleimplementation, the pulse-detection circuit 504 includes atransimpedance amplifier (TIA) 510, a gain circuit 512, a comparator514, and a time-to-digital converter (TDC) 516.

The TIA 510 may be configured to receive an electrical-current signalfrom the APD 502 and produce a voltage signal that corresponds to thereceived electrical-current signal. For example, in response to areceived optical pulse, the APD 502 may produce a current pulsecorresponding to the optical pulse. The TIA 510 may receive the currentpulse from the APD 502 and produce a voltage pulse that corresponds tothe received current pulse. The TIA 510 may also act as an electronicfilter. For example, the TIA 510 may be configured as a low-pass filterthat removes or attenuates high-frequency electrical noise byattenuating signals above a particular frequency (e.g., above 1 MHz, 10MHz, 20 MHz, 50 MHz, 100 MHz, 200 MHz, or any other suitable frequency).

The gain circuit 512 may be configured to amplify a voltage signal. Asan example, the gain circuit 512 may include one or morevoltage-amplification stages that amplify a voltage signal received fromthe TIA 510. For example, the gain circuit 512 may receive a voltagepulse from the TIA 510, and the gain circuit 512 may amplify the voltagepulse by any suitable amount, such as for example, by a gain ofapproximately 3 dB, 10 dB, 20 dB, 30 dB, 40 dB, or 50 dB. Additionally,the gain circuit 512 may also act as an electronic filter configured toremove or attenuate electrical noise.

The comparator 514 may be configured to receive a voltage signal fromthe TIA 510 or the gain circuit 512 and produce an electrical-edgesignal (e.g., a rising edge or a falling edge) when the received voltagesignal rises above or falls below a particular threshold voltage V_(T).As an example, when a received voltage rises above V_(T), the comparator514 may produce a rising-edge digital-voltage signal (e.g., a signalthat steps from approximately 0 V to approximately 2.5 V, 3.3 V, 5 V, orany other suitable digital-high level). As another example, when areceived voltage falls below V_(T), the comparator 514 may produce afalling-edge digital-voltage signal (e.g., a signal that steps down fromapproximately 2.5 V, 3.3 V, 5 V, or any other suitable digital-highlevel to approximately 0 V). The voltage signal received by thecomparator 514 may be received from the TIA 510 or the gain circuit 512and may correspond to an electrical-current signal generated by the APD502. For example, the voltage signal received by the comparator 514 mayinclude a voltage pulse that corresponds to an electrical-current pulseproduced by the APD 502 in response to receiving an optical pulse. Thevoltage signal received by the comparator 514 may be an analog signal,and an electrical-edge signal produced by the comparator 514 may be adigital signal.

The time-to-digital converter (TDC) 516 may be configured to receive anelectrical-edge signal from the comparator 514 and determine an intervalof time between emission of a pulse of light by the light source andreceipt of the electrical-edge signal. The output of the TDC 516 may bea numerical value that corresponds to the time interval determined bythe TDC 516. In some implementations, the TDC 516 has an internalcounter or clock with any suitable period, such as for example, 5 ps, 10ps, 15 ps, 20 ps, 30 ps, 50 ps, 100 ps, 0.5 ns, 1 ns, 2 ns, 5 ns, or 10ns. The TDC 516 for example may have an internal counter or clock with a20 ps period, and the TDC 516 may determine that an interval of timebetween emission and receipt of a pulse is equal to 25,000 time periods,which corresponds to a time interval of approximately 0.5 microseconds.Referring back to FIG. 1, the TDC 516 may send the numerical value“25000” to a processor or controller 150 of the lidar system 100, whichmay include a processor configured to determine a distance from thelidar system 100 to the target 130 based at least in part on an intervalof time determined by a TDC 516. The processor may receive a numericalvalue (e.g., “25000”) from the TDC 516 and, based on the received value,the processor may determine the distance from the lidar system 100 to atarget 130.

Using Acoustic Signals to Focus the Lidar System

FIG. 12 is a block diagram of an example computing system 600 forgenerating control signals for a lidar system based on acoustic signals.Generally speaking, the functionality of the computing system 600 can beimplemented in a dedicated device, multiple dedicated devices, or one ormore devices performing other functions in a lidar system or in anautonomous vehicle. As one example, the computing system 600 can beimplemented in the vehicle controller 372 depicted in FIG. 9. As anotherexample, the computing system 600 can be implemented in the controller150 or 306. As yet another example, components 610 and 612 can beprovided as a standalone acoustic source localization module, while thecomponents 612 and 614 can be provided as a separate module foranalyzing sound patterns. Further, some of the functionality of thecomputing system 600 can be implemented in a network server with whichthe vehicle controller 372 can communicate with a wireless communicationlink.

In this example implementation, the computing system 600 iscommunicatively coupled to the microphone array 392 (or the audiosensors 159A-C of FIG. 1, or any other suitable audio sensors), thelidar system 100, and the vehicle components 390 (see FIGS. 1 and 9).

The computing system 600 can include one or more processors 602, whichcan include one or more CPUs, one or more GPUs, or other types ofprocessors. The computing system 600 also can include a non-transitorymemory unit 604 storing software instructions that implement alocalization module 610 and a sound pattern analysis module 612. Thelocalization module 610 can implement any suitable acoustic localizationtechniques based on triangulation, trilateration, etc. The localizationmodule 610 also can provide a description of the detected acousticsignal in any suitable format and provide the description to the soundpattern analysis module 612, which in turn can implement any suitableanalog or digital signal processing techniques for comparing acousticsignatures to acoustic signals.

In addition to software instructions, the memory 604 can store a mapping614 that specifies how various sound patterns are mapped to variousactions to be applied to the lidar system. The sound pattern analysismodule 612 can utilize the mapping 614 when analyzing acoustic signals.The mapping 614 can be implemented as one or more tables of a relationaldatabase, for example, or in any other suitable manner. The mapping 614can include descriptions of sound patterns, or acoustic signatures, forvarious situations which an autonomous vehicle can encounter duringoperation. The descriptions can be stored in any suitable digital oranalog format. Some of the examples of acoustic signatures can includethe sound of tires screeching during abrupt breaking, the sound of thehorn of a typical car, the sound of the horn of a large truck, the soundof a bell at a railroad crossing, the sound of a whistle at a pedestriancrossing, the sound of an impact during collision, various sounds forsirens of different types (e.g., ambulance, police, fire truck), etc.

For each acoustic signature, the mapping 614 can specify an action or asequence of actions to be performed at the lidar system. For example,when the acoustic signature is that of a car abruptly breaking, and whenthe localization module 610 indicates that the abrupt breaking occurs infront of the autonomous vehicle in which the system 600 operates, theprescribed action may include increasing information density directlyahead of the autonomous vehicle at the lower portion of the verticalfield of regard. It is noted that the sound of breaking may come from asource not currently visible to the lidar system (e.g., from a sourceseveral vehicles ahead of the autonomous vehicle), and that bycollecting additional information within the region of interest 652, thelidar system can more quickly detect that the vehicle immediately aheadof the autonomous vehicle is also abruptly breaking, for example, andcause the vehicle controller to take appropriate defensive action.

As a more particular example, FIG. 13 illustrates an example region ofinterest 652 within a field of regard 650, which the mapping 614 canspecify as a part of the action corresponding to the acoustic signatureof abrupt breaking in front of the autonomous vehicle. The mapping 614can specify for the region of interest 652 a certain angular width, acertain angular height, a certain size, or a certain positioningrelative of the center of the field of regard 650. Further, the mapping614 can indicate the period of time during which the region of interest652 should remain active. Still further, the mapping 614 can indicatewhether the region of interest 652 moves across the field of regard 650or remains static within the field of regard 650. The angular width,angular height, size, or positioning of the region of interest 652 maybe determined based on lidar data (e.g., point-cloud data) or data fromanother sensor (e.g., a camera). For example, an acoustic sourcelocalization system may determine the location of a source of anacoustic signal, and point-cloud data or a camera image may be used todetermine that an object of interest (e.g., a vehicle in the distance)is the likely source of the acoustic signal. The size or location of theregion of interest 652 may be specified to encompass all or part of theobject of interest.

FIG. 14 illustrates another example region of interest 654 within thefield of regard 650. According to the mapping 614, the region ofinterest 654 can correspond to a different acoustic signature, such asthat of a police siren. Unlike the region of interest 652, the region ofinterest 654 moves across the field of regard 650 at a certain speedspecified by the mapping 614. The regions 652 and 654 also may differ insize and initial positioning within the field of regard 650, asillustrated in FIGS. 13 and 14. In some implementations, a region ofinterest 654 may be transferred from one lidar system to another lidarsystem. For example, the source of an acoustic signal associated withthe region of interest 654 may move across a first field of regard andinto a second field of regard that partially overlaps or is adjacent tothe first field of regard, and a controller may move the correspondingregion of interest 654 from the first field of regard to the secondfield of regard. In FIG. 14, the lidar system 100 may move the region ofinterest 654 across the field of regard 650 in accordance with themovement of the source of the acoustic signal. The source of theacoustic signal may move outside the field of regard 650 or may move atleast partially within a second field of regard of a second lidarsystem. The region of interest 654 may then be transferred to the secondfield of regard so that the second lidar system is used to track themovement of the source of the acoustic signal across the second field ofregard.

For each region of interest, the lidar system 100 can increase the pulserate, reduce scan speed, and/or modify the scan pattern to increaseinformation density in the region of interest. In some implementations,the mapping 614 can specify which operational parameters the lidarsystem 100 is to modify when scanning the region of interest. In otherimplementations, it is the controller 150 in the lidar system 100 thatdetermines how the region of interest should be scanned relative toother regions of the field of regard. As one example, the lidar system100 can increase the pulse rate when scanning the region of interest, sothat the pulse rate is higher for the region of interest relative to theremainder of the field of regard. As another example, the lidar system100 can decrease the scan speed when scanning the region of interest. Atthe same pulse rate, decreasing the scan speed results in higher pixeldensity. More generally, the lidar system 100 can focus on regions ofinterest to obtain more data or more accurate data by varying one ormore of the pulse rate, scan speed, scan pattern (e.g., by retracing theregion of interest without retracing the other regions), etc.

Next, FIG. 15 depicts a flow diagram of an example method 700 formodifying operation of a lidar system in view of acoustic signals. Forconvenience, the method 700 is discussed with reference to the computingsystem 600 discussed above, but in general the method 700 can beimplemented in any suitable system.

The method 700 begins at block 702, where an acoustic signal with anamplitude exceeding a certain threshold value is detected. The thresholdvalue can be selected so as to exceed the ambient noise level. Forexample, the threshold in various implementations can be 90 decibel, 100decibel, 110 decibel, etc. In some implementations, the computing system600 dynamically adjusts the threshold value in view of the ambient noiselevel. For example, the computing system 600 can decrease the thresholdvalue at night to improve sensitivity to acoustic signals, and increasethe threshold value during the day (or in busy places) to reduce thenumber of false positives.

The probable source of the acoustic signal is localized at block 704. Asdiscussed above, any suitable technique such as triangulation,trilateration, etc. can be used to estimate the direction and, whenpossible, the distance to the source of the signal. Then, at block 706,the acoustic signal can be analyzed to determine the likely type of thesource: e.g., a breaking car, an ambulance, a police car.

At block 708, one or more operational parameters of the lidar system aremodified in view of the determined probable source and its location. Tothis end, the computing system 600 can identify one or several regionsof interest within the field of regard. As discussed above, the one ormore operational parameters can include pulse rate, scan rate, scanpattern, etc. The results of processing the acoustic signal at block 704and 708 can be used in conjunction with other signals to modify the scanof a field of regard. For example, the lidar system can receive othersignals, such as those based on the results of image processingperformed on point clouds generated during previous scans, or those fromthe vehicle controller executing a maneuver, and modify one or moreoperational parameters in view of multiple signals. As a more specificexample, the lidar system can receive, from the vehicle controller, arequest to increase pixel density in a certain region of interest withinthe field of regard. The request may be related to the upcoming leftturn, for example. At substantially the same time, the lidar system canreceive a request to increase pixel density in a different region ofinterest determined based on acoustic signal. Depending on theimplementation, the lidar system can prioritize one region of interestover the other region of interest in view of the sources of the request,attempt to focus on both regions of interest, determine that neitherregion of interest should be the subject of a focus, etc.

General Considerations

In some cases, a computing device may be used to implement variousmodules, circuits, systems, methods, or algorithm steps disclosedherein. As an example, all or part of a module, circuit, system, method,or algorithm disclosed herein may be implemented or performed by ageneral-purpose single- or multi-chip processor, a digital signalprocessor (DSP), an ASIC, a FPGA, any other suitable programmable-logicdevice, discrete gate or transistor logic, discrete hardware components,or any suitable combination thereof. A general-purpose processor may bea microprocessor, or, any conventional processor, controller,microcontroller, or state machine. A processor may also be implementedas a combination of computing devices, e.g., a combination of a DSP anda microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

In particular embodiments, one or more implementations of the subjectmatter described herein may be implemented as one or more computerprograms (e.g., one or more modules of computer-program instructionsencoded or stored on a computer-readable non-transitory storage medium).As an example, the steps of a method or algorithm disclosed herein maybe implemented in a processor-executable software module which mayreside on a computer-readable non-transitory storage medium. Inparticular embodiments, a computer-readable non-transitory storagemedium may include any suitable storage medium that may be used to storeor transfer computer software and that may be accessed by a computersystem. Herein, a computer-readable non-transitory storage medium ormedia may include one or more semiconductor-based or other integratedcircuits (ICs) (such, as for example, field-programmable gate arrays(FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs),hybrid hard drives (HHDs), optical discs (e.g., compact discs (CDs),CD-ROM, digital versatile discs (DVDs), blue-ray discs, or laser discs),optical disc drives (ODDs), magneto-optical discs, magneto-opticaldrives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes,flash memories, solid-state drives (SSDs), RAM, RAM-drives, ROM, SECUREDIGITAL cards or drives, any other suitable computer-readablenon-transitory storage media, or any suitable combination of two or moreof these, where appropriate. A computer-readable non-transitory storagemedium may be volatile, non-volatile, or a combination of volatile andnon-volatile, where appropriate.

In some cases, certain features described herein in the context ofseparate implementations may also be combined and implemented in asingle implementation. Conversely, various features that are describedin the context of a single implementation may also be implemented inmultiple implementations separately or in any suitable sub-combination.Moreover, although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination may in some cases be excised from thecombination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

While operations may be depicted in the drawings as occurring in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order shown or in sequentialorder, or that all operations be performed. Further, the drawings mayschematically depict one more example processes or methods in the formof a flow diagram or a sequence diagram. However, other operations thatare not depicted may be incorporated in the example processes or methodsthat are schematically illustrated. For example, one or more additionaloperations may be performed before, after, simultaneously with, orbetween any of the illustrated operations. Moreover, one or moreoperations depicted in a diagram may be repeated, where appropriate.Additionally, operations depicted in a diagram may be performed in anysuitable order. Furthermore, although particular components, devices, orsystems are described herein as carrying out particular operations, anysuitable combination of any suitable components, devices, or systems maybe used to carry out any suitable operation or combination ofoperations. In certain circumstances, multitasking or parallelprocessing operations may be performed. Moreover, the separation ofvarious system components in the implementations described herein shouldnot be understood as requiring such separation in all implementations,and it should be understood that the described program components andsystems may be integrated together in a single software product orpackaged into multiple software products.

Various implementations have been described in connection with theaccompanying drawings. However, it should be understood that the figuresmay not necessarily be drawn to scale. As an example, distances orangles depicted in the figures are illustrative and may not necessarilybear an exact relationship to actual dimensions or layout of the devicesillustrated.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes or illustrates respective embodimentsherein as including particular components, elements, functions,operations, or steps, any of these embodiments may include anycombination or permutation of any of the components, elements,functions, operations, or steps described or illustrated anywhere hereinthat a person having ordinary skill in the art would comprehend.

The term “or” as used herein is to be interpreted as an inclusive ormeaning any one or any combination, unless expressly indicated otherwiseor indicated otherwise by context. Therefore, herein, the expression “Aor B” means “A, B, or both A and B.” As another example, herein, “A, Bor C” means at least one of the following: A; B; C; A and B; A and C; Band C; A, B and C. An exception to this definition will occur if acombination of elements, devices, steps, or operations is in some wayinherently mutually exclusive.

As used herein, words of approximation such as, without limitation,“approximately, “substantially,” or “about” refer to a condition thatwhen so modified is understood to not necessarily be absolute or perfectbut would be considered close enough to those of ordinary skill in theart to warrant designating the condition as being present. The extent towhich the description may vary will depend on how great a change can beinstituted and still have one of ordinary skill in the art recognize themodified feature as having the required characteristics or capabilitiesof the unmodified feature. In general, but subject to the precedingdiscussion, a numerical value herein that is modified by a word ofapproximation such as “approximately” may vary from the stated value by±0.5%, ±1%, ±2%, ±3%, ±4%, ±5%, ±10%, ±12%, or ±15%.

As used herein, the terms “first,” “second,” “third,” etc. may be usedas labels for nouns that they precede, and these terms may notnecessarily imply a particular ordering (e.g., a particular spatial,temporal, or logical ordering). As an example, a system may be describedas determining a “first result” and a “second result,” and the terms“first” and “second” may not necessarily imply that the first result isdetermined before the second result.

As used herein, the terms “based on” and “based at least in part on” maybe used to describe or present one or more factors that affect adetermination, and these terms may not exclude additional factors thatmay affect a determination. A determination may be based solely on thosefactors which are presented or may be based at least in part on thosefactors. The phrase “determine A based on B” indicates that B is afactor that affects the determination of A. In some instances, otherfactors may also contribute to the determination of A. In otherinstances, A may be determined based solely on B.

What is claimed is:
 1. A system comprising: an acoustic sourcelocalization system configured to (i) detect an acoustic signal and (ii)determine a location of a source of the acoustic signal relative to thesystem; a lidar system including: a light source configured to emitpulses of light, a scanner configured to scan a field of regard of thelidar system according to a scan pattern, and a detector configured todetect light from some of the light pulses scattered by one or moreremote targets to generate respective pixels in a point cloudrepresenting objects within the field of regard; and a controllercommunicatively coupled to the acoustic source localization system andthe lidar system, the controller configured to (i) receive, from theacoustic source localization system, an indication of the determinedlocation, (ii) identify a region of interest within the field of regardof the lidar system, the region of interest including the determinedlocation of the source of the acoustic signal, and (iii) cause the lidarsystem to obtain more data within the region of interest relative toother regions within the field of regard.
 2. The system of claim 1,wherein the controller is further configured to (iv) determine, usingindications received from the acoustic source localization system, amovement of the source of the acoustic signal across the field ofregard, (v) follow the movement by continuously modifying the region ofinterest, so that the region of interest moves across the field ofregard in accordance with the movement of the source of the acousticsignal, and (vi) cause the lidar system to obtain more data at differentrespective positions of the moving region of interest.
 3. The system ofclaim 2, further comprising a second lidar system configured to scan asecond field of regard, wherein the controller is further configured to(vii) determine that the source of the acoustic signal has moved atleast partially within the second field of regard, (viii) transfer theregion of interest to the second field of regard, and (ix) follow themovement of the source of the acoustic signal so that the region ofinterest moves across the second field of regard in accordance with themovement of the source of the acoustic signal.
 4. The system of claim 1,wherein the controller provides a signal to the lidar system indicatinga size of the region of interest and a duration of a time intervalduring which the lidar system is to obtain more data relative to otherregions within the field of regard.
 5. The system of claim 4, whereinthe controller is further configured to process a pattern of thedetected acoustic signal to determine a likely type of an object orevent to which the acoustic signal corresponds, and select at least oneof the size of the region of interest and the duration of the timeinterval based at least in part on the determined likely type of theobject or event.
 6. The system of claim 1, wherein to obtain more datawithin the region of interest, the lidar system increases a pulse rateat which the light source emits the pulses of light.
 7. The system ofclaim 1, wherein to obtain more data within the region of interest, thelidar system decreases a scan rate when the scanner scans the region ofinterest.
 8. The system of claim 1, wherein to obtain more data withinthe region of interest, the lidar system modifies the scan pattern toincrease pixel density within the region of interest.
 9. A method forscanning a field of regard of a lidar system, the method comprising:emitting light pulses by a light source in the lidar system; scanning,by a scanner in the lidar system, the field of regard of the lidarsystem in accordance with a certain scan pattern, including directingthe light pulses toward different points within the field of regard;receiving an acoustic signal; determining a location of a source of theacoustic signal relative to the lidar system; identifying a region ofinterest within the field of regard based at least in part on thedetermined location of the source of the sound; modifying at least oneoperational parameter of the lidar system to obtain more data within theidentified region of interest, relative to other regions within thefield of regard; and detecting light from at least some of the lightpulses scattered by one or more remote targets to generate respectivepixels in a point cloud representing objects within the field of regard.10. The method of claim 8, further comprising: determining a movement ofthe source of the acoustic signal across the field of regard; followingthe movement by continuously modifying the region of interest, so thatthe region of interest moves across the field of regard in accordancewith the movement of the source of the acoustic signal, and modifyingthe at least one operational parameter of the lidar system to obtainmore data at different respective positions of the moving region ofinterest.
 11. The method of claim 10, further comprising: determiningthat the source of the acoustic signal has moved at least partiallywithin a second field of regard of a second lidar system; transferringthe region of interest to the second field of regard; and following themovement of the source of the acoustic signal so that the region ofinterest moves across the second field of regard in accordance with themovement of the source of the acoustic signal.
 12. The method of claim9, wherein identifying the region of interest within the field of regardincludes determining a size of the region and a duration of a timeinterval during which the lidar system is to obtain more data relativeto other regions within the field of regard.
 13. The method of claim 12,wherein determining the size of the region and the duration of a timeinterval includes determining a likely type of an object or event towhich the acoustic signal corresponds, and select at least one of thesize of the region of interest and the duration of the time intervalbased at least in part on the determined likely type of the object orevent.
 14. The method of claim 9, wherein modifying the at least oneoperational parameter of the lidar system to obtain more data within theidentified region of interest includes increasing a pulse rate at whichthe light source emits the pulses of light.
 15. The method of claim 9,wherein modifying the at least one operational parameter of the lidarsystem to obtain more data within the identified region of interestincludes decreasing a scan rate when the scanner scans the region ofinterest.
 16. The method of claim 9, wherein modifying the at least oneoperational parameter of the lidar system to obtain more data within theidentified region of interest includes modifying the scan pattern toincrease pixel density within the region of interest.
 17. An autonomousvehicle comprising: one or more components configured to maneuver theautonomous vehicle; an acoustic source localization system configured to(i) detect an acoustic signal and (ii) determine a location of a sourceof the acoustic signal relative to the autonomous vehicle; a lidarsystem including: a light source configured to emit pulses of light, ascanner configured to scan a field of regard of the lidar systemaccording to a scan pattern, and a detector configured to detect lightfrom some of the light pulses scattered by one or more remote targets togenerate respective pixels in a point cloud representing objects withinthe field of regard; and a controller communicatively coupled to the oneor more components configured to maneuver the autonomous vehicle, theacoustic source localization system, and the lidar system, thecontroller configured to (i) receive, from the acoustic sourcelocalization system, an indication of the determined location, (ii)identify a region of interest within the field of regard of the lidarsystem, the region of interest including the determined location of thesource of the acoustic signal, (iii) cause the lidar system to obtainmore data within the region of interest relative to other regions withinthe field of regard, and (iv) maneuver the autonomous vehicle based atleast in part on the point cloud generated by the lidar system.
 18. Theautonomous vehicle of claim 17, wherein the controller is furtherconfigured to (iv) determine, using indications received from theacoustic source localization system, a movement of the source of theacoustic signal across the field of regard, (v) follow the movement bycontinuously modifying the region of interest, so that the region ofinterest moves across the field of regard in accordance with themovement of the source of the acoustic signal, and (vi) cause the lidarsystem to obtain more data at different respective positions of themoving region of interest.
 19. The autonomous vehicle of claim 17,wherein the controller provides a signal to the lidar system indicatinga size of the region of interest and a duration of a time intervalduring which the lidar system is to obtain more data relative to otherregions within the field of regard.
 20. The autonomous vehicle of claim19, wherein the controller is further configured to process a pattern ofthe detected acoustic signal to determine a likely type of an object orevent to which the acoustic signal corresponds, and select at least oneof the size of the region of interest and the duration of the timeinterval based at least in part on the determined likely type of theobject or event.
 21. The autonomous vehicle of claim 17, wherein toobtain more data within the region of interest, the lidar systemincreases a pulse rate at which the light source emits the pulses oflight.
 22. The autonomous vehicle of claim 17, wherein to obtain moredata within the region of interest, the lidar system decreases a scanrate when the scanner scans the region of interest.
 23. The autonomousvehicle of claim 17, wherein to obtain more data within the region ofinterest, the lidar system modifies the scan pattern to increase pixeldensity within the region of interest.